ARKit and ARCore: Mobile AR Development Guide

ARKit and ARCore bring augmented reality to the billions of smartphones already in people's pockets. Unlike headset-based XR, mobile AR requires no specialized hardware purchase — your app's AR features are available to anyone with a compatible iPhone or Android phone. This makes mobile AR the highest-reach XR platform by far.

The trade-off is limited immersion compared to headsets: mobile AR is screen-based, requires holding a phone, and lacks true depth perception. But for consumer product visualization, marketing campaigns, educational overlays, and navigation, mobile AR delivers compelling value at global scale.


Core AR Capabilities: ARKit vs. ARCore Feature Comparison

- World Tracking (SLAM): Both ARKit and ARCore provide 6DoF world tracking — the phone's position and orientation in 3D space is tracked without GPS - Plane Detection: Both detect horizontal and vertical surfaces. LiDAR-equipped iOS devices detect planes faster and more accurately - Light Estimation: Both provide ambient light intensity and color temperature. ARKit adds directional light estimation on Pro devices - Image Tracking: Both support detecting and tracking flat 2D images (product labels, posters, business cards) - Object Detection: ARKit supports detecting and tracking 3D objects scanned with the Object Scanner app - LiDAR Meshing: ARKit on Pro devices generates dense room meshes enabling occlusion and realistic object placement - People Occlusion: ARKit can occlude AR content behind real people using segmentation - Face Tracking: ARKit uses TrueDepth camera for precise face mesh tracking on iPhones with Face ID ## Setting Up Unity AR Foundation

Unity AR Foundation is the recommended cross-platform approach for mobile AR — build once, deploy to both iOS and Android.

- Create a new Unity project (URP template recommended for AR lighting) - Install AR Foundation, ARKit XR Plugin, and ARCore XR Plugin via Package Manager - Create an AR Session Origin with an AR Camera - Add an AR Session component to manage AR state - Add AR Plane Manager for horizontal/vertical surface detection - Add AR Raycast Manager for surface hit testing Test in Unity's AR Simulation mode (in-editor testing without a phone) by creating a simulated AR environment. For device testing, connect an iPhone (iOS 15+ required for ARKit 5) or Android phone (Android 7+ with ARCore support) and build from Unity.

Core AR Development Patterns

Placing Objects on Surfaces

The most fundamental mobile AR interaction: user taps the screen, a raycast detects the AR plane at that point, and a virtual object appears on the surface. Implement with ARRaycastManager.Raycast(), spawning a prefab at the hit pose. Use smooth placement with scale-in animation for polished feel.

Image Tracking

Image tracking detects a predefined image in the camera feed and tracks a virtual object relative to it. Create a reference image library in Unity with your target images and their physical sizes. Add an AR Tracked Image Manager and bind response logic for when images are detected, updated (tracking active), and lost.

LiDAR Meshing and Occlusion (iOS Pro)

AR Mesh Manager (AR Foundation) activates LiDAR meshing on supported devices. The mesh can be used for physics (virtual objects rolling on real surfaces), occlusion (virtual objects hidden behind real walls), and environmental interaction. Set mesh materials to invisible (depth-only) for seamless occlusion without visible geometry.

Native ARKit Development (Swift)

For apps targeting only iOS with deep platform integration, native ARKit in Swift/SwiftUI provides the most capability. ARSCNView (SceneKit) or ARView (RealityKit) are the main view types. ARConfiguration subclasses (ARWorldTrackingConfiguration, ARFaceTrackingConfiguration, ARBodyTrackingConfiguration) configure the AR session for specific use cases.

Performance Tips for Mobile AR

- Minimize shadow-casting lights — expensive on mobile GPUs - Use Compressed textures (ASTC on iOS, ETC2 on Android) - Keep polygon counts low: 50K–200K total scene polygons depending on device tier - Profile with Xcode Instruments (iOS) or Android GPU Inspector (Android) to identify GPU bottlenecks - Throttle feature detection rates on ARPlaneManager and ARRaycastManager when not actively needed ## Distribution: App Store and Google Play

Mobile AR apps submit to the Apple App Store and Google Play through standard app submission processes. ARKit apps require iOS 11+ (realistically iOS 15+ for modern features). ARCore apps require Android 7+ on supported devices. Both stores list device requirements that filter the install audience appropriately.