Skip to Content
Platform.Ontology

LiDAR and depth sensing

Gets a depth map or room mesh of what the camera sees.

Works without askingHardware and accessoriesSensorsAndroid and iPhoneWhile you use it

In one line

Can measure depth and room shape through the camera, on phones with the hardware

Gets a depth map or room mesh of what the camera sees.

Reveals the shape of rooms and faces. TrueDepth depth maps identify facial structure, so access is tied to the camera prompt.

What it can and can't do

What the user does: You tap Scan room or Measure, and point the camera around

It can

  • Build a depth map of what the camera sees
  • Make a 3D model of a room on iPhone Pro models with LiDAR, up to about 5 meters
  • Use front camera depth for faces (TrueDepth on iPhone)
  • Estimate depth from motion on Android without a special sensor

It cannot

  • Use depth without the camera prompt
  • See past about 5 meters with LiDAR
  • Get true time of flight depth on most Android phones

How it differs between platforms: iPhone Pro models have LiDAR and an API for it; Android has no standard LiDAR API and true depth sensors exist only on some models.

On Android and iPhone

AndroidiPhone
StatusPartialSupported
First versionAndroid 7.0 (API 24)iOS 11.0
Permission or entitlementCAMERA (depth through ARCore or Camera2 DEPTH16 on devices that have ToF)NSCameraUsageDescription
Protection leveldangerousprompt
How access is grantedpromptprompt
What the user seesCamera promptCamera prompt
How long it lastsUntil revokedUntil revoked
How to take it backSettings > Apps > Permissions > CameraSettings > Privacy & Security > Camera
Store policyNone listedNone listed
APIARCore Depth API (Google Play Services for AR), Camera2 DEPTH16, no standard LiDAR APIAVCaptureDepthDataOutput (iOS 11, TrueDepth), ARKit sceneDepth and ARMeshAnchor (LiDAR, iOS 14, Pro models), RoomPlan (iOS 16)
NotesARCore Raw Depth works without ToF using motion; true ToF only on selected devices.LiDAR on iPhone 12 Pro and later, iPad Pro; up to 5 m range.

Data you get

What comes back: Depth map
Shape: stream
How often: up to 30 to 60 fps

FieldTypeUnitRangePrecision
depthfloatm0 to 5 m (LiDAR)about 1 cm

Technical limits

  • No separate depth permission; camera permission covers it

The story

What it is

A depth sensor or depth estimate that tells the app how far things are in front of the camera.

For you

A measuring app instantly tells you how tall a door is, or a room scanner builds a 3D model as you walk around it. Nothing seems to be recorded, but your room shape is captured.

For makers

Depth makes AR objects hide behind real furniture, makes portrait mode sharp, and lets apps build 3D models of homes and bodies. Face depth data is sensitive because it can describe a face shape, so access is tightly limited.

Where you see it

TypeExampleApps
ar-vrA virtual ball bounces off your real sofa and rolls behind your table.Apple AR Quick Look
shoppingA home improvement app scans your room so it can suggest furniture that fits.None named
creativeA 3D scan app builds a model of a statue or a sneaker for sharing.Polycam
gamesAn AR game hides virtual enemies behind walls and corners in your real room.None named
securityA face unlock flow uses a depth map of the face, not just a photo, so a picture cannot fool it.Face ID
utilitiesA measuring app finds the width of a window.Measure
healthA fitness app counts reps by tracking the body in 3D.None named

Risks

  • 3D maps of your home
  • Face shape data
  • Models stored in the cloud

Android and iPhone in short

  • Android: Most phones use motion based depth from ARCore. A few have a true depth sensor.
  • iPhone: LiDAR is on iPhone 12 Pro and later and iPad Pro, with a range up to about 5 meters.

What the platform says

  • Apple: LiDAR Scanner. Measure the distance to objects and map rooms in 3D for AR, photography and measuring.

Good uses and abuse

Legitimate uses

  • Room scanning
  • Measuring apps
  • Face unlock like flows

Known abuse

  • Face geometry capture

Scores

ScoreValueMeaning
Sensitivity4 of 5How revealing the data is
Utility3 of 5How useful it is to honest apps
Friction2 of 5How much the user has to do
Fingerprint bits3Rough identifying entropy, when known

History

  • 2020-08, Android ARCore 1.18: Depth API (stereo from motion)
  • 2020-09, iPhone iOS 14: LiDAR scene reconstruction
  • 2022-09, iPhone iOS 16: RoomPlan room scanning

Same idea on the web

Sources

How well checked: Based on official documentation.

On this page