LiDAR and depth sensing
Gets a depth map or room mesh of what the camera sees.
In one line
Can measure depth and room shape through the camera, on phones with the hardware
Gets a depth map or room mesh of what the camera sees.
Reveals the shape of rooms and faces. TrueDepth depth maps identify facial structure, so access is tied to the camera prompt.
What it can and can't do
What the user does: You tap Scan room or Measure, and point the camera around
It can
- Build a depth map of what the camera sees
- Make a 3D model of a room on iPhone Pro models with LiDAR, up to about 5 meters
- Use front camera depth for faces (TrueDepth on iPhone)
- Estimate depth from motion on Android without a special sensor
It cannot
- Use depth without the camera prompt
- See past about 5 meters with LiDAR
- Get true time of flight depth on most Android phones
How it differs between platforms: iPhone Pro models have LiDAR and an API for it; Android has no standard LiDAR API and true depth sensors exist only on some models.
On Android and iPhone
| Android | iPhone | |
|---|---|---|
| Status | Partial | Supported |
| First version | Android 7.0 (API 24) | iOS 11.0 |
| Permission or entitlement | CAMERA (depth through ARCore or Camera2 DEPTH16 on devices that have ToF) | NSCameraUsageDescription |
| Protection level | dangerous | prompt |
| How access is granted | prompt | prompt |
| What the user sees | Camera prompt | Camera prompt |
| How long it lasts | Until revoked | Until revoked |
| How to take it back | Settings > Apps > Permissions > Camera | Settings > Privacy & Security > Camera |
| Store policy | None listed | None listed |
| API | ARCore Depth API (Google Play Services for AR), Camera2 DEPTH16, no standard LiDAR API | AVCaptureDepthDataOutput (iOS 11, TrueDepth), ARKit sceneDepth and ARMeshAnchor (LiDAR, iOS 14, Pro models), RoomPlan (iOS 16) |
| Notes | ARCore Raw Depth works without ToF using motion; true ToF only on selected devices. | LiDAR on iPhone 12 Pro and later, iPad Pro; up to 5 m range. |
Data you get
What comes back: Depth map
Shape: stream
How often: up to 30 to 60 fps
| Field | Type | Unit | Range | Precision |
|---|---|---|---|---|
depth | float | m | 0 to 5 m (LiDAR) | about 1 cm |
Technical limits
- No separate depth permission; camera permission covers it
The story
What it is
A depth sensor or depth estimate that tells the app how far things are in front of the camera.
For you
A measuring app instantly tells you how tall a door is, or a room scanner builds a 3D model as you walk around it. Nothing seems to be recorded, but your room shape is captured.
For makers
Depth makes AR objects hide behind real furniture, makes portrait mode sharp, and lets apps build 3D models of homes and bodies. Face depth data is sensitive because it can describe a face shape, so access is tightly limited.
Where you see it
| Type | Example | Apps |
|---|---|---|
| ar-vr | A virtual ball bounces off your real sofa and rolls behind your table. | Apple AR Quick Look |
| shopping | A home improvement app scans your room so it can suggest furniture that fits. | None named |
| creative | A 3D scan app builds a model of a statue or a sneaker for sharing. | Polycam |
| games | An AR game hides virtual enemies behind walls and corners in your real room. | None named |
| security | A face unlock flow uses a depth map of the face, not just a photo, so a picture cannot fool it. | Face ID |
| utilities | A measuring app finds the width of a window. | Measure |
| health | A fitness app counts reps by tracking the body in 3D. | None named |
Risks
- 3D maps of your home
- Face shape data
- Models stored in the cloud
Android and iPhone in short
- Android: Most phones use motion based depth from ARCore. A few have a true depth sensor.
- iPhone: LiDAR is on iPhone 12 Pro and later and iPad Pro, with a range up to about 5 meters.
What the platform says
- Apple: LiDAR Scanner. Measure the distance to objects and map rooms in 3D for AR, photography and measuring.
Good uses and abuse
Legitimate uses
- Room scanning
- Measuring apps
- Face unlock like flows
Known abuse
- Face geometry capture
Scores
| Score | Value | Meaning |
|---|---|---|
| Sensitivity | 4 of 5 | How revealing the data is |
| Utility | 3 of 5 | How useful it is to honest apps |
| Friction | 2 of 5 | How much the user has to do |
| Fingerprint bits | 3 | Rough identifying entropy, when known |
History
- 2020-08, Android ARCore 1.18: Depth API (stereo from motion)
- 2020-09, iPhone iOS 14: LiDAR scene reconstruction
- 2022-09, iPhone iOS 16: RoomPlan room scanning
Related capabilities
Related
- Augmented reality world tracking. Can see through the camera and map your room while the app is open
- Camera extensions and hardware camera controls. Can use phone camera modes such as night and HDR, after you allow the camera
Same idea on the web
- WebXR world sensing (hit test, anchors, depth, planes, camera). Can detect surfaces and depth in your room, and even see camera frames
Sources
How well checked: Based on official documentation.