Skip to Content
Platform.Ontology

Augmented reality world tracking

Maps the surroundings and places digital objects on surfaces.

Works without askingHardware and accessoriesSensorsAndroid and iPhoneWhile you use it

In one line

Can see through the camera and map your room while the app is open

Maps the surroundings and places digital objects on surfaces.

Builds a 3D map of a person's room and tracks where they look. Cloud and geospatial anchors can send visual data to the platform vendor.

What it can and can't do

What the user does: You tap Try it in your room or open a camera based AR feature

It can

  • Find floors and tables through the camera and place digital objects on them
  • Track where the phone is pointing
  • Use your location to pin objects to real places, if you allow it
  • Upload visual feature points to Google for cloud or Geospatial features (Android)

It cannot

  • Start without the camera prompt
  • Use your location for geo anchors unless you allow location
  • Send the room map off an iPhone by default; it stays on the device
  • Run on Android without Google Play Services for AR

How it differs between platforms: iPhone keeps the world map on the device; Android cloud and Geospatial features send data to Google.

On Android and iPhone

AndroidiPhone
StatusSupportedSupported
First versionAndroid 7.0 (API 24)iOS 11.0
Permission or entitlementCAMERA; ACCESS_FINE_LOCATION for Geospatial API; Google Play Services for ARNSCameraUsageDescription; NSLocationWhenInUseUsageDescription for geo anchoring
Protection leveldangerousprompt
How access is grantedpromptprompt
What the user seesCamera prompt; location prompt for GeospatialCamera prompt
How long it lastsUntil revokedUntil revoked
How to take it backSettings > Apps > PermissionsSettings > Privacy & Security > Camera
Store policyPlay: AR apps declare uses-feature and ARCore as required or optionalNone listed
APIARCore Session, Geospatial API, Cloud Anchors (data sent to Google)ARKit ARSession, ARWorldTrackingConfiguration, ARGeoTrackingConfiguration (iOS 14), RealityKit
NotesCloud and Geospatial features upload feature points and need ARCore Cloud ID.World mapping stays on device; iOS 26 adds spatial widgets and Shared World Anchors on visionOS.

Data you get

What comes back: Camera pose and anchors
Shape: stream
How often: 30 to 60 Hz

FieldTypeUnitRangePrecision
poseobject6 DoFunboundedabout 1 to 2 cm drift

Technical limits

  • Camera dot and permission apply on both

The story

What it is

The phone maps the room around you and keeps digital objects stuck to floors, tables and walls.

For you

You point your phone at the floor and a virtual dragon or couch appears and stays put as you walk around it. It feels magical, but the camera is on and the phone is looking at your room.

For makers

AR needs a live map of the room, which unlocks games, shopping previews and measuring tools. Cloud anchors let two people see the same object. Makers also learn the layout of homes and stores if the data leaves the phone.

Where you see it

TypeExampleApps
gamesYou throw Poke Balls at a Pikachu standing in your living room and the creature seems to sit on your floor.Pokemon GO
shoppingYou place a sofa in your lounge to see if it fits and what colour looks right.IKEA Place, Amazon
gamesA multiplayer AR game lets two friends see the same castle on a table and play at once.Minecraft Earth (retired)
mapsA walking app paints arrows on the street through the camera view.Google Maps Live View
educationA biology class lets students walk around a giant virtual heart.None named
creativeA face and world filter adds animated effects to a video.Snapchat
utilitiesA measuring app turns your phone into a tape measure.Measure

Risks

  • Camera sees your home
  • Shared maps can leak layouts
  • Battery drain

Android and iPhone in short

  • Android: Cloud and geospatial features upload camera feature points to Google and need an ARCore cloud ID.
  • iPhone: World mapping stays on the device. iOS 26 adds spatial widgets and shared world anchors on visionOS.

What the platform says

  • Apple: ARKit. Build augmented reality that tracks the real world and places virtual objects with precision.
  • Google: ARCore. Build augmented reality experiences that blend digital content with the real world.

Good uses and abuse

Legitimate uses

  • Furniture preview
  • Navigation
  • Games

Known abuse

  • Scanning private rooms
  • Sending visual maps to cloud

Scores

ScoreValueMeaning
Sensitivity4 of 5How revealing the data is
Utility3 of 5How useful it is to honest apps
Friction2 of 5How much the user has to do
Fingerprint bits3Rough identifying entropy, when known

History

  • 2017-06, iPhone iOS 11: ARKit introduced
  • 2018-03, Android ARCore 1.0 (Android 7): ARCore introduced
  • 2022-05, Android ARCore 1.31: Geospatial API

Same idea on the web

Sources

How well checked: Based on official documentation.

On this page