Skip to Content
Platform.Ontology

Text recognition and visual look up (Live Text, Visual Look Up, ML Kit)

Reads text in images and scans documents on device.

Works without askingOn-device AIVisionAndroid and iPhoneWhile you use it

In one line

Can pick out text in pictures and scan documents on your phone

Reads text in images and scans documents on device.

Apps can recognise text, scan pages and identify objects in photos. Visual Look Up and Live Text work inside the system apps with no code.

What it can and can't do

What the user does: You tap Scan document, point the camera at text, or open a photo

It can

  • Pick out the words in a photo or live camera view
  • Scan a document without the camera permission on Android, using Google's scanner
  • Look up landmarks and plants on iPhone, with Apple's servers if needed

It cannot

  • Use the live camera without you allowing the camera
  • Look at photos you did not open or share
  • Keep working fully offline on iPhone for landmark look up

How it differs between platforms: Both need a camera prompt for live scanning. Android's document scanner uses Google's screen, so it needs no camera permission.

On Android and iPhone

AndroidiPhone
StatusSupportedSupported
First versionAndroid 5.0 (API 21)iOS 13.0
Permission or entitlementCAMERA for live capture (dangerous)None for images; camera permission for live
Protection leveldangerousprompt
How access is grantedpromptprompt
What the user seesCamera permission for live; none for static imagesCamera prompt for live; none for photos
How long it lastsUntil revokedUntil revoked
How to take it backSettings > Privacy > CameraSettings > Privacy > Camera
Store policyNone listedNone listed
APIML Kit Text Recognition v2, ML Kit Document Scanner, Google LensVisionKit ImageAnalyzer, DataScannerViewController (iOS 16), VNRecognizeTextRequest
NotesML Kit Document Scanner needs no camera permission as it uses Play services UI.Visual Look Up uses Apple models and may call servers for landmarks.

Data you get

What comes back: Recognised text
Shape: object
How often: per image

FieldTypeUnitRangePrecision
textstringcharsper languagegood

Technical limits

  • Quality varies with language and lighting

The story

What it is

Reads text in pictures and scans documents on the phone.

For you

You point the camera at a page and can copy the words, or tap a plant in a photo to learn its name. It works on the phone, though some lookups ask a server.

For makers

Makers add scanning, receipts and ID capture without building their own system. It also turns photos into searchable data, which brings privacy questions.

Where you see it

TypeExampleApps
bankingA bank app reads a check or card number from a photo.Chase
productivityA scanner app turns a paper receipt into a text file.Adobe Scan, Notes
travelA traveler copies a hotel address from a sign.None named
shoppingA shopping app reads a barcode or label.None named
educationA student scans a page into notes.None named
accessibilityA person with low vision has a sign read aloud.Seeing AI
gamesAn AR game reads text from a real poster to unlock a clue.None named

Risks

  • Captures documents and IDs
  • Some lookups call servers

Android and iPhone in short

  • Android: Camera permission for live scans. The ML Kit Document Scanner needs no camera permission.
  • iPhone: Camera prompt for live scans. Visual Look Up may call Apple servers for landmarks.

What the platform says

Good uses and abuse

Legitimate uses

  • Scanning
  • Accessibility

Known abuse

  • Document capture

Scores

ScoreValueMeaning
Sensitivity3 of 5How revealing the data is
Utility4 of 5How useful it is to honest apps
Friction1 of 5How much the user has to do
Fingerprint bits1Rough identifying entropy, when known

History

  • 2019-09, iPhone iOS 13: Vision text recognition
  • 2022-09, iPhone iOS 16: Live Text in apps and DataScanner

Same idea on the web

Sources

How well checked: Based on official documentation.

On this page