Navigating the Future: The Evolution of Spatial Computing
A deep dive into the technologies driving spatial computing and their real-world applications
Table of Contents
Spatial Data Capture: Sensor Networks, IoT, and Data Fusion
2. Cameras
5. Radar
10. Magnetometers
12. Optical Sensing
Spatial Computing Infrastructure
Spatial Computing Applications
Introduction
Spatial computing integrates the digital layer with the physical world, enabling users to interact with digital information in their environment. The technology supports various modes of interaction, such as visual, auditory, and haptic feedback, and is designed to be unobtrusive. It aims to blur the lines between digital and physical spaces, enhancing everyday tasks by providing real-time information and interactions. This approach aligns with the concept of ubiquitous computing, where technology seamlessly integrates into daily activities.
Figure 1: Spatial Computing is an evolving 3D-centric form of computing that, at its core, uses AI, Computer Vision and extended reality to blend virtual experiences into the physical world making almost every surface a spatial interface. It combines software, hardware, data/information, and connectivity [1]
Spatial computing relies on three key elements: 3D data, location, and time. 3D data enriches content by adding spatial depth, although this requires substantial computing power and storage. Location precision is vital for accurately overlaying digital information onto the physical world, as seen in industrial digital twins where millimeter accuracy is crucial. Time allows for tracking changes, analyzing historical trends, and forecasting future scenarios, making it essential for dynamic interactions within spatial computing environments. [2]
The U.S. spatial computing market size was estimated at USD 27.59 billion in 2023 and is predicted to be worth around USD 244.40 billion by 2034, at a CAGR of 21.9% from 2024 to 2034. [3]
Challenges in Spatial Computing Development and Adoption
The challenges of spatial computing involve managing the complexities of integrating new technologies with existing systems. Key challenges in enterprise tech adoption include:
Hardware Compatibility: Ensuring new devices work seamlessly with existing IT infrastructure.
High Computing Demands: Spatial computing applications often require significant computing power and storage.
Network Performance: Ensuring sufficient bandwidth and low latency for real-time applications.
Security and Compliance: Protecting sensitive data and ensuring new technologies meet regulatory requirements.
Skill Gaps: Addressing the need for IT professionals skilled in emerging spatial technologies.
From the developer side, there are also challenges in the development of spatial computing technologies, which include:
User Experience: Issues like motion sickness and interface design remain significant hurdles, despite advancements in devices like Apple's Vision Pro that claim to offer better user interaction.
Accessibility: Spatial computing is still not widely accessible due to high costs, the need for specialized technical skills, and infrastructure requirements.
Integration: Effective integration with other devices and systems is crucial, yet challenging, as it requires spatial computing devices to seamlessly connect with different databases and interfaces.
Standards and Development: The lack of established standards and regulations for hardware and software development in this field complicates the creation of universally compatible and secure applications.
Spatial Computing Strategy of the Hyper Scalers
Apple: Apple's spatial computing features have been integrated over time, culminating in technologies that enhance real-world and digital interactions. These include GPS functionality, ARKit for augmented reality, the U1 chip using ultra-wideband (UWB) for precise spatial awareness, LiDAR sensors for depth perception, and continuous enhancements in Apple Maps for detailed spatial data. Apple is leveraging its Vision Pro headset, equipped with VisionOS and various developer tools, to pioneer new experiences in augmented reality (AR) and virtual reality (VR). Their approach is marked by a mix of high-end hardware offerings and a software ecosystem designed to foster app development and create immersive user experiences. Similar to its strategy with previous products, Apple is taking a controlled approach with its Vision Pro headset, maintaining strict control over the hardware and the ecosystem.
Meta: Meta has continued to invest heavily in XR, releasing products like the Quest 3 headset and Ray-Ban Meta smart glasses. Meta's strategy has evolved to a more open ecosystem, allowing third-party hardware integration and expanding the utility of its platforms. This shift is part of Meta's broader goal to bridge its social media base with more extensive XR functionalities.
Google: While Google's involvement in XR has been more subdued following the discontinuation of Daydream and Glass, they continue to impact the spatial computing space through products like ARCore and Earth Engine. Google's potential collaboration with Samsung on new XR headsets could further leverage ARCore's capabilities, enhancing their competitive stance in the XR market by creating more sophisticated and integrated AR experiences.
Nvidia: NVIDIA is strategically expanding its presence in the XR sector with its Omniverse platform, which connects 3D applications through the cloud using NVIDIA GPUs. This enables remote collaboration in both 2D and 3D, positioning NVIDIA as a central hub for XR software usage across various industries. The platform leverages the USD file format to facilitate interoperability among different applications, emphasizing NVIDIA’s commitment to using open standards to enhance its dominance in professional graphics and extend its capabilities into AR and VR.
Microsoft: Microsoft has scaled back its XR ambitions, focusing now on integrating its software offerings with other platforms' hardware. This includes bringing its Microsoft 365 suite to headsets like the Meta Quest 3 and Apple's Vision Pro, moving away from direct competition in the hardware space. In the meanwhile, Microsoft continueously targets business applications with its HoloLens 2, positioning it primarily as an enterprise solution rather than a consumer product.
Spatial Computing Technology Stack
The spatial computing tech stack integrates general and specific components essential for immersive digital interactions. General components include foundational hardware like processors and standard networking, security, and cloud computing services that support data management and connectivity. Spatial-specific components encompass sensors, cameras, and IoT devices for data capture; software tools and SDKs tailored for spatial data processing and 3D visualization; and user-oriented hardware such as headsets.
Figure 2: Spatial Computing Tech Stack by Clarice Qiu
Figure 3: Spatial Computing Market Mapping by Clarice Qiu
Spatial Data Capture: Sensor Networks, IoT, and Data Fusion
Description: Sensor networks and IoT devices collect diverse real-time data from the physical world, enabling monitoring, control, and decision-making. Integrating these with other data sources—like geospatial data, digital twins, and crowdsourced inputs—enhances spatial awareness. Sensor fusion combines data from multiple sources (e.g., LiDAR, GPS, cameras) to improve accuracy and context, supporting more reliable spatial applications.
Sensors: Devices like cameras, LiDAR, GPS, and other environmental sensors that gather various types of spatial data.
IoT Devices: Network-connected devices that collect and transmit data from their environments.
Data Acquisition Systems: Hardware and software that facilitate the collection, initial processing, and storage of sensor data.
Data Integration Tools: Software that combines data from different sources, providing a unified view and analysis capabilities.
Communication Networks: Systems that ensure data is reliably transmitted from the field to data processing centers or cloud storage.
Spatial Computing Sensors
Spatial computing relies on various sensors to collect and process data from the physical environment, enabling systems to understand and interact with the world in three dimensions. Here are some of the most commonly used sensors in spatial computing:
1. LiDAR (Light Detection and Ranging)
Function: LiDAR uses laser pulses to measure distances by timing the reflection of the pulses from surfaces. This creates a detailed 3D map of the environment, known as a point cloud.
Applications: Used in autonomous vehicles, drones, robotics, and environmental mapping for creating accurate 3D models of surroundings.
3D LiDAR vs. 4D LiDAR: Understanding the Differences
3D LiDAR (Light Detection and Ranging):
Functionality: 3D LiDAR uses laser pulses to measure the distance to objects in the environment, creating a three-dimensional map or point cloud. This map includes detailed spatial information about the shape and size of objects, as well as their positions in the environment.
Applications: 3D LiDAR is widely used in autonomous vehicles, robotics, and mapping applications, where it helps systems understand the structure and layout of their surroundings. It provides static information about the environment at a given moment, which is essential for tasks like obstacle detection and navigation.
Output: The output of 3D LiDAR is a collection of points in space (a point cloud), which represent the 3D surfaces of objects in the environment.
4D LiDAR:
Functionality: 4D LiDAR builds on the capabilities of 3D LiDAR by adding the dimension of motion. In addition to measuring the distance to objects and creating a 3D map, 4D LiDAR also captures the velocity (speed and direction) of objects in real-time. This is achieved through the use of advanced Doppler effect techniques, which allow the system to detect changes in the frequency of the returned laser signal as objects move.
Applications: 4D LiDAR is particularly valuable in dynamic environments, such as in autonomous vehicles, where understanding the movement of objects (like pedestrians, other vehicles, or cyclists) is crucial for safe navigation. It enables more advanced predictive modeling, allowing systems to anticipate and respond to changes in the environment more effectively.
Output: The output of 4D LiDAR includes both the 3D position data and the velocity data of each detected object. This additional dimension provides a richer dataset that can be used for more complex decision-making processes in real-time.
Startup Example: Aeva
Summary: Aeva develops advanced 4D LiDAR sensors, including Aeries II and Atlas, which utilize Frequency Modulated Continuous Wave (FMCW) technology to provide instant velocity and range detection for autonomous vehicles, robotics, and industrial applications. These sensors offer high resolution, long-range detection, and resistance to interference, enabling safer and more precise autonomous navigation. Aeva’s technology is used in automotive, transportation, and industrial sectors to improve perception and decision-making in complex environments.
Head Office: Mountain View, CA
Funding: founded in 2016, $560M raised
2. Cameras
Function: Cameras capture visual information in 2D, which can be processed to infer depth, movement, and object recognition.
Types:
RGB Cameras: Capture standard color images.
Depth Cameras: Capture depth information along with color, often used in conjunction with other sensors.
Applications: Used in augmented reality (AR), virtual reality (VR), facial recognition, and object detection.
Company Example: Insta360
Summary: Insta360 provides a range of cameras for 360-degree and action photography, including the Insta360 X3, Pro 2, and Titan, along with AI-powered accessories like the Insta360 Flow stabilizer. The technology features 8K 360° video, FlowState stabilization, and AI tools for dynamic tracking and editing. Use cases include virtual tours, sports, filmmaking, real estate, and immersive VR content creation, offering creative and professional solutions for diverse industries.
Head Office: Shenzhen, Guangdong
Funding: founded in 2014, IPO in 2019
3. IMUs (Inertial Measurement Units)
Function: IMUs measure acceleration, rotation, and magnetic field strength using accelerometers, gyroscopes, and magnetometers.
Applications: Critical in navigation and orientation for drones, smartphones, VR headsets, and other devices that require precise movement tracking.
Startup Example: Aceinna
Summary: Aceinna specializes in advanced sensing technologies, particularly focusing on inertial systems, current sensors, and flow sensors. They provide high-precision inertial measurement units (IMUs) that are critical for navigation and positioning in various applications such as autonomous vehicles and drones. Their current sensors are integral to energy management systems, offering high accuracy and low noise performance. Additionally, Aceinna's flow sensors are utilized in medical and industrial applications for precise fluid measurements. This breadth of products underlines Aceinna’s role in enhancing sensor technology and supporting the development of automation and control systems across multiple industries.
Head Office: Massachusetts, United States
Funding: founded in 2017, $50M raised
4. GPS (Global Positioning System)
Function: GPS provides geographic location data by communicating with satellites.
Applications: Used in mapping, navigation, and any application that requires precise location tracking, such as autonomous vehicles and outdoor AR applications.
Startup Example: zephr
Summary: Zephr is pioneering advanced GPS technology that offers enhanced accuracy up to less than 1 meter for mobile devices and wearables. This breakthrough is achieved through a networked solution that interlinks satellite measurements across devices, significantly improving location precision and reliability while maintaining strong user privacy. Zephr's technology, delivered via an SDK, promises to unlock new potentials across diverse fields such as augmented reality, gaming, mobility, logistics, and safety, among others.
Head Office: San Jose, CA
Funding: Founded in 2022, $3.5 M seed round raised
Figure 4: How the Zephr Technology works
5. Radar
Function: Radar uses radio waves to detect the distance, velocity, and angle of objects.
Applications: Widely used in autonomous vehicles for navigation and obstacle detection, weather monitoring to track storms and precipitation, and in aviation for aircraft detection and tracking.
Synthetic Aperture Radar (SAR) is an active remote sensing technology that uses radio waves emitted from a moving vehicle to detect objects and landscapes. The ability of these waves to penetrate obstacles like leaves enables the estimation of tree structures and heights using different wavelengths. SAR operates independently of light or weather conditions because it does not rely on ambient light, allowing for effective imaging through clouds.
Startup Example: Capella Space
Summary: Capella Space specializes in Synthetic Aperture Radar (SAR) technology, delivering high-resolution satellite imagery capable of penetrating cloud cover for reliable Earth observation under any weather conditions. Their technology supports a variety of applications, including environmental monitoring, infrastructure inspection, and emergency response. Capella provides actionable insights for diverse industries such as intelligence, insurance, maritime, and humanitarian aid, emphasizing rapid data delivery and high-cadence global monitoring.
Head Office: San Francisco, CA
Funding: founded in 2015, $239M raised
6. Ultrasonic Sensors
Function: These sensors measure distances by emitting sound waves and timing how long they take to reflect back from surfaces.
Applications: Common in robotics for obstacle detection and avoidance, as well as in parking assistance systems in vehicles.
7. Infrared Sensors
Function: Infrared sensors detect heat signatures or measure distance based on infrared light reflection.
Applications: Used in night vision, motion detection, eye tracking, and proximity sensing in various devices, including home security systems and robotics.
8. Time-of-Flight (ToF) Sensors
Function: ToF sensors measure the time it takes for a light pulse to travel to an object and back, providing depth information.
Applications: Used in AR/VR, smartphones for facial recognition, and gesture tracking systems.
9. Pressure Sensors
Function: Measure the force applied to a surface, often used in combination with other sensors for touch-based interfaces.
Applications: Used in touchscreen devices, robotics (for grip and force feedback), and interactive installations.
10. Magnetometers
Function: Measure magnetic fields, often used for orientation and heading information.
Applications: Common in smartphones for compass functionality, and in combination with IMUs for navigation in spatial computing systems.
11. Environmental Sensors
Function: These sensors measure various environmental factors like temperature, humidity, and air quality.
Applications: Used in smart homes, environmental monitoring, and some industrial applications where environmental conditions affect spatial operations.
12. Optical Sensing
Function: Optical sensors utilize light to detect changes in an environment or measure certain characteristics. They range from simple photodiodes that detect light intensity to complex systems like LIDAR and structured light sensors.
Applications: Extensively used in machine vision, security systems, medical imaging, and barcode scanning. Optical sensing is pivotal in AR/VR for environmental mapping, user interaction through eye-tracking, and gesture recognition.
Startup Example: NIL Technologies
Summary: NIL Technology specializes in advanced optical solutions for various applications, including AR/MR displays, 3D sensing, and consumer electronics. Their offerings encompass design, prototyping, and manufacturing of diffractive optical elements, metalenses, and waveguides, enhancing device performance in industries like automotive and IoT. Their technology enables compact and efficient optical components for high-precision applications, supporting innovative uses in machine vision and interactive displays.
Head Office: Denmark
Funding: founded in 2006, $70M raised
Watch more:
These sensors work together in spatial computing systems to provide comprehensive data about the environment, enabling more precise and dynamic interaction with the physical world.
Data Acquisition Systems
Startup Example: Move AI
Summary: Move AI specializes in advanced markerless motion capture technology that transforms 2D video into 3D motion data using AI, physics, and biomechanical models. Their offerings, including Move One and Move Multi-Cam, support high-quality human motion capture in various environments, enabling up to 22 people to be tracked simultaneously without suits or studios. This technology is utilized across gaming, film, and live activations, simplifying 3D animation workflows and enhancing real-time interactive experiences.
Head Office: London, UK
Funding: founded in 2019, $10M in seed funding raised
Watch More:
Data Integration and Management
Startup Example: Wherobots
Summary: Wherobots specializes in spatial intelligence for large-scale data operations, offering products like Wherobots Cloud, Spatial Catalog, WherobotsDB, and WherobotsAI. These tools facilitate spatial ETL, analytics, and AI-driven insights across various industries, supporting data operations in a cloud-native lake-house architecture. Their technology is especially beneficial for companies dealing with massive geospatial datasets, providing solutions that enable efficient data management, advanced spatial queries, and strategic decision-making.
Head Office: Scottsdale, AZ
Funding: founded in 2022, $5.5M raised
Figure 5: Wherobots spatial datasets
Spatial Data Processing, Analytics and Generation
Spatial Analytics and Context-Aware Computing
Description: Spatial Analytics and Context-Aware Computing together enhance spatial computing by enabling systems to analyze and respond to spatial and contextual data in real-time. Spatial Analytics extracts insights from spatial data, such as traffic patterns or environmental conditions, while Context-Aware Computing uses these insights to adapt system behavior based on real-time context, like location, time, and user activity.
Examples: GIS-Based Urban Planning Tools, Smart Assistants
Startup Example: Basemark
Summary: Basemark specializes in augmented reality software and services for the automotive industry, offering products like Rocksolid AR, a tool designed for creating immersive AR experiences tailored to automotive applications. Their technology focuses on enhancing driver safety and comfort by integrating AR into vehicle displays, such as HUDs and instrument clusters, improving real-time contextual guidance and alerts. Basemark's solutions cater to major automotive clients, helping them deliver advanced AR features that improve navigation and driving dynamics.
Head Office: Helsinki Finland
Funding: founded in 2015, $36.3M raised
Watch More:
Spatial User Interfaces (SUIs)
Description: SUIs enable users to interact with digital content in a three-dimensional space using gestures, voice commands, or motion tracking.
Examples: Gesture control systems, VR gloves, voice-activated interfaces like Amazon Echo.
Startup Example: cognitive3D
Summary: Cognitive3D specializes in advanced analytics for VR, AR, and MR environments, providing tools that help businesses understand user behavior in three-dimensional spaces. Their platform, compatible with major 3D engines like Unity and Unreal, offers features such as detailed session replay, dynamic object tracking, and biometric monitoring to optimize user interactions and engagement. This technology is widely used in training simulations, consumer research, academic studies, and entertainment, aiding developers in refining and advancing their immersive applications.
Head Office: Vancouver, Canada
Funding: founded in 2015, $2.5M raised
SLAM (Simultaneous Localization and Mapping) & Spatial Mapping
Description: Simultaneous Localization and Mapping (SLAM) enables devices to build and update a map of an unknown environment while determining their position within it. This process is vital for autonomous navigation, AR, and VR, allowing devices to interact accurately with the physical world by creating detailed spatial maps.
Examples:
Self-Driving Cars: Use SLAM to navigate complex environments, recognize obstacles, and make real-time decisions.
Drones: Employ SLAM for autonomous flight and obstacle avoidance in unstructured environments.
AR Navigation Apps: Use SLAM to guide users indoors or in complex outdoor settings without GPS.
Startup Example: slamcore
Summary: Slamcore develops advanced spatial intelligence solutions for the intralogistics industry, focusing on enhancing the autonomy of vehicles like forklifts, AGVs (Autonomous Guided Vehicles), and AMRs (Autonomous Mobile Robots). Their main offering, Slamcore Aware, uses cutting-edge visual-SLAM (Simultaneous Localization and Mapping) technology and sensor fusion to provide real-time, precise navigation and object detection without needing extensive infrastructure. This technology is especially beneficial in dynamic environments such as warehouses, where it improves operational efficiency and safety.
Head Office: London, UK
Funding: Founded in 2016, $27.4M raised
3D Model Generation, Management, and Trading
Description: 3D model generation involves the creation of digital three-dimensional representations of objects or environments. Management refers to organizing and maintaining these models, ensuring they are updated and accurate. Trading encompasses the exchange or sale of 3D models in marketplaces, where they can be utilized in various applications such as gaming, animation, and virtual reality.
Examples: 3D models are extensively used in video game development, architectural visualizations, and online marketplaces like TurboSquid and CGTrader where designers sell and share content.
3D Modeling
In 3D model generation, several key technologies are commonly used: photogrammetry, which captures complex details from real-world objects using multiple photographs; 3D scanning, which collects precise data on the shape and appearance of objects; and Computer-Aided Design (CAD), which allows for the manual creation of detailed models and blueprints. Additionally, advanced techniques like Neural Radiance Fields (NeRFs) and procedural generation are used for creating high-fidelity and complex environments by synthesizing photorealistic scenes or landscapes from minimal input data. Each method has unique advantages, catering to different needs in areas ranging from entertainment to engineering.
Photogrammetry: This technique involves capturing photographs from multiple angles and using them to create 3D models. It's highly accurate for reconstructing real-world objects and environments but requires complex post-processing and can be resource-intensive.
Neural Radiance Fields (NeRFs): NeRFs use a fully connected neural network to model continuous volumetric scenes. By leveraging multiple 2D images of a static scene from different angles, NeRFs can synthesize novel view points of that scene with impressive detail and photorealism. This method is computationally intensive but produces high-quality 3D representations.
Procedural Generation: A method that uses algorithms to automatically create data or content, such as textures, models, or entire game worlds. This approach can produce vast amounts of content with little input data, though it may lack the intricate details that methods like photogrammetry can provide.
Gaussian Splatting: This method is used in image processing to render and blur points in 3D data, employing a Gaussian function to smooth transitions. It's primarily useful for anti-aliasing and softening image features in visualization tasks.
Each of these technologies serves different needs in 3D model generation, with NeRFs standing out for high-quality synthesis, photogrammetry for accurate real-world reconstructions, procedural generation for autonomously generate vast and varied 3D content, and Gaussian splatting for its utility in graphics smoothing and visualization.
Startup Example: Polycam
Summary: Polycam offers a comprehensive suite of tools for 3D model generation using advanced technologies like LiDAR, photogrammetry, and Gaussian splatting, accessible via mobile apps and web platforms. Besides, Polycam's AI-driven texture generation and precise drone photogrammetry enable detailed aerial data capture. These features are essential for industries such as architecture, urban planning, and virtual reality, allowing for the creation of precise and highly accurate 3D models. Additionally, Polycam includes collaborative tools that enhance sharing and management of 3D models, promoting efficiency and creativity in handling 3D spatial data.
Head Office: San Francisco, CA
Funding: founded in 2020, $22.1M raised
3D Model Trading and Management
Startup Example: CGTrader
Summary: CGTrader is a leading online platform for 3D modeling and digital content creation, offering a diverse marketplace with over two million 3D models. It caters to various industries such as gaming, advertising, and architecture with products for AR/VR, 3D printing, and animation. The platform also supports designers through tools like CGDream for AI-generated 3D models and Modelry for custom 3D modeling solutions. CGTrader is ideal for both purchasing pre-made models and hiring freelancers for bespoke projects.
Head Office: Lithuania
Funding: founded in 2012, $12.1 M raised
Spatial Computing Infrastructure
Spatial computing infrastructure encompasses the hardware, software, and network systems that enable interaction and navigation within three-dimensional spaces. Here are the major components:
Hardware:
Edge Devices: Sensors, cameras, AR/VR headsets, and other IoT devices.
Processors: CPUs, GPUs, and specialized processing units for real-time data handling.
Software:
Development Tools: SDKs, 3D engines, and libraries for application development.
AI/ML Technologies: Algorithms and frameworks for intelligent data analysis and decision-making.
Data Management:
Spatial Data Management: Systems for managing spatial information, including acquisition, storage, and retrieval.
Synthetic Data: Generation and integration of artificial data to enhance model training and simulation.
Data Pipelines: Architecture for data ingestion, processing, and analytics, ensuring data flow between devices and systems.
Connectivity & Network:
Infrastructure to support seamless data transmission and accessibility across devices.
Security & Privacy:
Technologies and protocols to protect data integrity and user privacy in spatial environments.
Cloud Computing:
Services and platforms for scalable processing, storage, and access, central to managing extensive data loads and complex computations.
Hardware
Startup Example: Magic Leap
Summary: Magic Leap specializes in Mixed Reality (MR) technology through its Magic Leap 2 headset, which merges the physical and digital worlds, enhancing user interaction with both. This MR headset incorporates advanced spatial computing, precise eye-tracking, and high-resolution sensors to support a range of applications in sectors like healthcare, manufacturing, and architecture. Its immersive capabilities enable professionals to interact with data and digital objects in real-time, improving workflow efficiency and engagement.
Head Office: Plantation, FL
Funding: founded in 2010, raised $3.98B
Startup Example: Light Field Lab
Summary: Light Field Lab is revolutionizing visual experiences with its SolidLight technology, which creates high-resolution, holographic displays that bring digital content into the real world as three-dimensional objects. Their system includes modular video wall panels, advanced computational hardware, and WaveTracer™ software for real-time rendering, targeting use cases in advertising, entertainment, and virtual collaboration. This technology allows for transformative, lifelike digital interactions without the need for traditional screens or VR headsets.
Head Office: San Jose, CA
Funding: founded in 2017, $85M raised
Figure 6: Solidlight system by Light Field Lab
Development Tools
For spatial computing, several SDKs (Software Development Kits) are available to help developers create applications in areas such as augmented reality (AR), virtual reality (VR), and mixed reality (MR). Some popular SDKs include:
ARCore by Google - ARCore is Google’s augmented reality SDK that supports Android, iOS, Unity, and Web platforms. It features motion tracking, environmental understanding for detecting various surfaces, and anchors for stable object placement. Additionally, ARCore includes depth sensing and light estimation to enhance the realism of AR experiences.
Unity- Provides tools for AR, VR, and game development across multiple platforms.
Unreal Engine - Offers robust support for VR and AR development with high-fidelity visuals.
Vuforia - Known for its AR development capabilities across mobile and digital eyewear.
Meta offers a collection of tools for AR/VR/MR development.
Spark AR: Spark AR is Meta's augmented reality platform that enables creators to design AR experiences integrated into Facebook and Instagram. This SDK offers tools for building effects, animations, and interactive AR content.
Meta Spatial SDK: Developed for mixed reality environments, the Meta Spatial SDK helps transform 2D apps with MR and VR elements, ensuring cross-device compatibility.
XR All-in-One SDK: A comprehensive SDK package combines various Meta SDKs for developing virtual and mixed reality applications. It offers a broad range of features for immersive experience creation, including advanced rendering and social capabilities, by bundling specialized SDKs like Core, Audio, Haptics, and Interaction into one unified toolkit.
Apple provides a range of tools for AR/VR/MR development through its various platforms, particularly targeting augmented reality applications. Key technologies and tools include:
ARKit: This is Apple's framework for building augmented reality experiences on iOS devices. It provides features such as motion tracking, environmental understanding, and light estimation to create immersive AR applications.
RealityKit: Another powerful tool from Apple that allows for more realistic and complex AR experiences. It provides a high-performance rendering engine, a physics engine, and simple ways to animate objects.
Reality Composer and Reality Converter: These are apps that help developers create and manage AR content. Reality Composer is for building AR experiences, while Reality Converter is used to convert, view, and customize 3D objects on macOS.
VisionOS: designed for Apple Vision Pro, merges the physical and digital worlds to offer an immersive XR experience. It features advanced interaction models and sensor integration, supporting unique spatial interactions that are distinct from iOS and macOS. This OS represents a significant advancement in spatial computing environments.
Company Example: Unreal Engine
Summary: Unreal Engine delivers high-end graphics and performance, primarily known for powering AAA games and cinematic experiences. Its robust toolset includes advanced rendering capabilities, a full-fledged physics engine, and Blueprints for visual scripting. Unreal's high fidelity makes it a top choice for projects requiring top-tier visual quality, such as detailed simulations, VR experiences, and interactive media in sectors like film production, architectural visualization, and automotive design.
Head Office: Cary, NC
Founding: founded in 1995, part of Epic Game
Company Example: Unity
Summary: Unity is a versatile real-time 3D development platform that streamlines creation across various devices and platforms. It's widely embraced for mobile and indie game development due to its ease of use and extensive support for multiple formats. Beyond gaming, Unity finds applications in industries like automotive, architecture, and film for interactive visualizations and virtual simulations. Its comprehensive ecosystem aids developers with tools, assets, and a supportive community.
Head Office: San Francisco, CA
Funding: founded in 2004, IPO in 2020
AI/ML Technologies
AI/ ML technologies play a crucial role in enhancing spatial computing by providing advanced capabilities for understanding, interpreting, and interacting with the spatial world. Key AI technologies used in spatial computing include:
Computer Vision: For object recognition, environment understanding, facial expression recognition, hand gesture recognition and real-time scene analysis.
Machine Learning & Deep Learning: Enhances accuracy in spatial data interpretation, predictive modeling, and natural user interaction.
Natural Language Processing (NLP): Enables voice commands and conversational interfaces in AR/VR/MR environments.
Reinforcement Learning: Used for navigation, path planning, and autonomous decision-making in dynamic spatial environments.
Generative AI: For creating synthetic environments, digital twins, and realistic 3D content.
Spatial Intelligence: Employs AI to process and analyze three-dimensional spatial data, enabling systems to navigate and operate within complex environments more effectively.
These AI technologies enable more immersive, intuitive, and responsive spatial experiences across various applications such as AR, VR, MR, robotics, autonomous navigation, and smart environments.
Computer Vision
Computer vision enables machines to interpret and understand the visual world through digital images and videos, providing the necessary data about the environment. Spatial computing uses this visual data to interact with and manipulate the physical space. It applies the insights gained from computer vision to perform tasks such as navigation, object recognition, and 3D modeling, making spatial computing systems more intuitive and interactive.
Startup Example: Groundlight
Summary: Groundlight.ai offers innovative computer vision solutions, enabling industries like manufacturing, retail, and facility management to leverage AI for real-time image analysis without the need for extensive pre-existing datasets. Their technology suite includes the Groundlight Hub and tools for developers like a Python SDK and ROS, simplifying integration into existing systems. These products are used for applications such as process control, safety monitoring, and operational inspections, providing immediate, scalable answers to visual queries.
Head Office: Seattle, WA
Funding: founded in 2019, $10M seed round raised
Spatial AI
Spatial intelligence refers to the ability of AI systems to understand and process the spatial relationships between objects in an environment. It involves recognizing the position, orientation, and relationship of objects in a three-dimensional space. This capability allows AI to navigate, manipulate, or interact with physical objects and environments effectively. Spatial intelligence is crucial in applications such as robotics, autonomous vehicles, augmented reality, and geographic information systems, where understanding the physical layout and dynamics of spaces is essential.
Spatial AI is the application of spatial intelligence principles through artificial intelligence systems. It leverages algorithms and machine learning techniques to interpret and navigate these spaces, making it possible for machines and systems to interact intelligently with their physical surroundings.
Four components of Spatial AI [4]
Spatial data
Spatial intelligence for AI systems to interpret, navigate, and manipulate objects of the physical world
AI algorithms such as machine learning and deep learning techniques specifically designed to handle spatial data and extract insights.
Data fusion techniques to combine visual data (images, videos) with depth data (LiDAR) to create a richer 3D understanding.
Figure 7: Spatial AI brain: an imagining of how the representation and processing graph structures of a general Spatial AI system might map to a graph processor. The key elements we identify are the real-time processing loop, the graph-based map store, and blocks which interface with sensors and output actuators. Note that we envision additional ‘close to the sensor’ processing built into visual sensors, aiming to reduce the data bandwidth (eventually in two directions) between the main processor and cameras, which will generally be located some distance away. [5]
Startup Example: World Labs
Summary: World Labs, initiated by AI visionary Fei-Fei Li, advances spatial intelligence by developing AI capable of understanding and interacting with the 3D world. The team's focus on Large World Models (LWMs) enhances AI's interaction in real and virtual environments, targeting transformative applications in fields like robotics and immersive simulations. This technology equips AI to more accurately perceive, navigate, and operate within complex spaces, aiming to redefine the capabilities of AI systems across various industries.
Head Office: San Francisco, CA
Funding: founded in 2024, $230M raised
Synthetic Data
Synthetic data plays a critical role in spatial computing by providing artificially generated data that mimics real-world scenarios. This is particularly valuable for training machine learning models when real-world data is scarce, expensive, or sensitive. In spatial computing, synthetic data can simulate complex environments and interactions, allowing systems to learn and adapt without the risks or costs associated with real-environment training. This approach enhances the robustness and accuracy of spatial computing applications such as autonomous driving, augmented reality, and robotics.
Startup Example: synthesis.ai
Summary: Synthesis AI provides synthetic data generation for enhancing computer vision and perception AI applications. Their offerings include highly detailed, privacy-compliant synthetic datasets that simulate complex real-world scenarios for ID verification, driver and pedestrian monitoring, and AR/VR applications. This technology accelerates the training of AI models by providing diverse and perfectly labeled data, thereby reducing biases and enhancing the robustness of AI systems in various industries.
Head Office: San Francisco, CA
Funding: founded in 2019, $21.7M raised
Mobile Device Management (MDM)
Startup Example: ArborXR
Summary: ArborXR offers a management platform for AR and VR devices, tailored for businesses that leverage immersive technologies. The platform streamlines the deployment and operation of XR applications across various industries by providing tools for remote management, application installation, updates, and user experience customization through kiosk mode. This makes it an effective solution for sectors engaged in training and educational activities utilizing augmented and virtual reality.
Head Office: Oklahoma, US
Funding: founded in 2020, $12M raised
Figure 8: Admin Remote View for process monitoring
Spatial Computing Applications
Figure 9: Key spatial computing applications by Clarice Qiu
Extended Reality (XR) Experience
In spatial computing, XR (Extended Reality) functions as a key technology that bridges the gap between the digital and physical worlds, enabling interactive experiences that are visually and contextually integrated with the user's environment. XR encompasses VR (Virtual Reality), AR (Augmented Reality), and MR (Mixed Reality), each providing unique ways to augment, simulate, or merge real and virtual scenes. This integration allows for applications such as immersive training environments, enhanced retail shopping experiences, and sophisticated remote collaboration in real-time.
Figure 10: The term XR includes AR, MR, VR, and any technology that blends the physical and the digital world. [6]
Augmented Reality (AR)
Description: AR overlays digital content onto the real world, enhancing the user’s perception of their environment. It blends virtual elements with the physical world in real time.
Examples: Pokémon Go, IKEA Place app, and AR navigation systems.
Sector to Watch: Mobile AR
Mobile augmented reality (AR) integrates digital content with the physical world through the camera of a mobile device. It overlays digital information such as images, text, or interactive data onto the real-world environment as seen through the device’s screen. This overlay dynamically updates as the user moves their device, providing real-time interaction with both physical and virtual elements.
Use Cases of Mobile AR:
Retail: Consumers can use mobile AR to compare product features, prices, and reviews by scanning items with their smartphones.
Education: AR brings educational content to life with interactive elements, enhancing engagement and learning.
Field Services: Technicians and engineers can see schematics or repair instructions overlaid directly onto the equipment they are servicing.
Logistics: In warehouses, AR can guide workers to specific locations and provide picking instructions, improving efficiency.
Advantages of Mobile AR:
Enhanced User Engagement: Merging digital and physical views, AR creates compelling user experiences that are both informative and interactive.
Accessibility: Utilizes everyday smartphones and tablets, making it widely accessible without the need for additional specialized hardware.
Versatility: Applicable across various fields from gaming to professional training, offering practical and entertainment value.
Advanced 3D Modeling: Incorporating technologies like Apple’s LiDAR in mobile devices allows for better 3D model creation and enhanced AR experiences by improving depth sensing and environmental understanding.
Challenges of Mobile AR:
Data Accuracy: High accuracy in data capture is essential, especially in environments where precise information overlay is critical.
User Experience Design: AR requires unique design approaches compared to traditional apps to ensure interactions are both intuitive and useful.
Technical Limitations: Reliance on the device’s camera and processing power can restrict the complexity and smoothness of AR applications.
Mobile AR, leveraging existing mobile technology, significantly enhances user interaction across various sectors by blending digital interactions with the physical world, thereby increasing both utility and engagement in everyday applications.
Startup Example: Hololink
Summary: Hololink offers an AR platform that enables brands and agencies to create interactive augmented reality experiences to boost engagement and conversions. Their user-friendly AR editor allows for easy customization without coding, making it accessible for creative marketing campaigns. The platform integrates AI-driven analytics to measure attention, memory, and conversion rates, and is used for immersive marketing, education, and tourism experiences. Hololink has been adopted by clients worldwide for innovative digital marketing strategies.
Head Office: Denmark
Funding: founded in 2019, $151k raised
Startup Example: Vyry by Phygtl
Summary: Vyry is a platform that redefines digital engagement for younger generations, merging the physical and digital worlds to create immersive social experiences. It leverages state-of-the-art technologies to allow users to transform everyday objects into interactive experiences using a mobile app, emphasizing user autonomy over data and digital assets. Targeted towards students and young creators, Vyry offers a collaborative space where creativity meets social interaction, fostering a community of co-creators rather than mere consumers of digital content.
Head Office: Foster City, CA
Funding: founded in 2022 , early-stage VC funded
Startup Example: Scandit
Summary: Scandit specializes in smart data capture solutions using mobile technology. Their offerings include barcode scanning, ID scanning, and augmented reality applications designed to enhance operational efficiency in various industries such as retail, logistics, and healthcare. Scandit's platform utilizes advanced computer vision technology integrated with mobile devices, enabling real-time data capture and processing. This technology supports a range of applications from inventory management to customer engagement, showcasing its versatility across multiple sectors.
Head Office: Zürich, Switzerland
Funding: founded in 2009, $273M raised
Virtual Reality (VR)
Description: VR immerses users in a completely virtual environment, typically through headsets that block out the physical world and replace it with a digital one.
Examples: Oculus Quest 2, HTC Vive, and VR Chat
Startup Example: ORamaVR
Summary: ORamaVR specializes in medical XR training, offering the MAGES platform for creating, recording, and publishing XR simulations. Their technology targets educators and learners in healthcare institutions and med-tech companies, enabling immersive training to reduce errors and improve patient outcomes. ORamaVR's solutions are designed to enhance the competency and proficiency of medical professionals through realistic simulations, facilitated by AI analytics and personalized assessment tools.
Head Office: Switzerland
Funding: founded in 2016, € 14.4 M raised
Startup Example: Merge EDU
Summary: Merge EDU integrates AR and VR technologies to enhance STEM education, offering digital teaching aids, simulations, and interactive holograms. Its platform is used in classrooms to facilitate hands-on learning and engagement, supporting visual-spatial learning styles with curriculum-aligned content for science and STEM subjects. Merge EDU's tools are designed to help students grasp complex concepts through immersive experiences, making learning interactive and accessible both in schools and at home.
Head Office: San Antonio, Texas
Funding: founded in 2014, $11.49M raised
Mixed Reality (MR)
Description: MR is a hybrid of AR and VR, where virtual objects are not just overlaid on the real world but also interact with it. MR environments are fully aware of the physical surroundings, allowing virtual objects to coexist and interact with real-world objects.
Examples: Microsoft HoloLens, Magic Leap, Meta Quest
Startup Example: SynergyXR
Summary: SynergyXR provides an Extended Reality (XR) platform that enhances technical training, collaboration, and communication for industries by creating immersive learning and working environments. It supports a range of XR applications without the need for coding, making it accessible for non-technical users. This platform is ideal for industries looking to scale their training processes or enhance their operational workflows with virtual reality, offering solutions that are both innovative and practical.
Head Office: Denmark
Funding: founded in 2008, $5.18M raised
Geospatial Mapping
Description: This involves using data and sensors to create detailed maps and models of physical spaces. These maps can be used for navigation, urban planning, and various other applications where understanding the spatial layout is critical.
Examples: Google Earth, LiDAR-based mapping for autonomous vehicles.
Startup Example: ROCK Robotic
Summary: ROCK Robotic offers integrated LiDAR solutions for 3D mapping and spatial data collection. Their product lineup includes the R3 Pro V2 LiDAR scanner, ROCK SLAM Dock V2 for handheld mapping, and various mounts for drones and mobile vehicles. Their software solutions, ROCK Cloud and ROCK Desktop, provide tools for data processing, visualization, and collaboration. These technologies are used in industries like surveying, construction, and forestry for accurate topographic mapping, asset inspection, and environmental monitoring.
Head Office: Denver, CO
Funding: founded in 2020, self-funded
Startup Example: Hivemapper
Summary: Hivemapper is a dynamic platform that leverages a global network to generate fresh, high-quality map data for various industries. It offers products like the Map Image API and Map Features API, which provide access to street-level imagery and precise geolocated traffic infrastructure details. Hivemapper also features tools like Scout for advanced location monitoring and Bursts for on-demand, targeted map data collection. The technology integrates specially designed dashcams and AI to constantly update and refine map data, supporting applications in navigation, automotive, logistics, and more.
Head Office: San Francisco, CA
Funding: founded in 2015, $21M total raised
Startup Example: Planet Labs
Summary: Planet Labs provides an array of products and services tailored to various industries, enhancing decision-making through daily satellite imagery and data analytics. Their offerings include Planet Monitoring for real-time observation, Planet Tasking for on-demand satellite imaging, and analytic feeds for actionable insights. These solutions cater to sectors like agriculture, energy, government, and sustainability, offering precise monitoring and management tools to track changes and manage resources effectively.
Head Office: San Francisco, CA
Funding: founded in 2010, $573.9 M raised
Digital Twin
Description: A digital twin is a virtual replica of a physical system, object, or process that can be used for real-time simulation, monitoring, and analysis. This technology allows for continuous updates and synchronization between the virtual and physical models, enabling predictive maintenance, system optimization, and deeper insights into performance and potential issues.
Examples: Digital twins are widely used in industries such as manufacturing for monitoring production lines, in urban planning for city management, and in healthcare for simulating medical treatments.
Startup Example: Matterport
Summary: Matterport provides a 3D capture platform for creating digital twins of real-world spaces, enabling immersive virtual tours and detailed spatial analytics. Their offerings include a range of cameras (from smartphones to Pro3 lidar) and AI-powered tools for insights, measurements, and layouts. Key use cases span real estate, construction, retail, and facilities management, helping to optimize operations, enhance marketing, and streamline workflows. The platform integrates with software like Autodesk and Procore, making it versatile for various industries.
Head Office: Sunnyvale, CA
Funding: Founded in 2011, Matterport was acquired by CoStar Group for $1.6B on Apr 22, 2024.
Company Example: PTC
Summary: PTC offers digital transformation solutions across the product lifecycle, including 3D CAD (Creo), PLM (Windchill), IIoT (ThingWorx), and AR (Vuforia). Their technology integrates real-time data, digital twins, and augmented reality to improve product design, manufacturing, and service processes. PTC solutions are widely used in manufacturing, engineering, and service optimization to drive innovation, reduce costs, and enhance product quality and customer experiences. They serve industries such as automotive, aerospace, electronics, and more.
Head Office: Boston, MA
Funding: 1985 founded, 2013 IPO
Robotics and Autonomous Systems
Description: These systems use spatial computing to navigate and interact with their environment. Autonomous vehicles, drones, and industrial robots rely heavily on spatial computing for tasks such as navigation, object recognition, and interaction with physical objects.
Examples: Autonomous drones, self-driving cars, warehouse robots.
Startup Example: Field AI
Summary: Field AI develops Field Foundation Models™ (FFMs), enabling robots to operate autonomously in unstructured environments. Their technology supports various vehicles and sensors, ensuring resilience and adaptability. Use cases span industries like construction, oil & gas, and safety, enhancing efficiency, safety, and precision through AI-driven autonomy. With roots in DARPA-winning teams, Field AI integrates embodied intelligence and robotics for scalable, real-world applications.
Head Office: Mission Viejo, CA
Funding: founded in 2023
Startup Example: Vicarious Surgical
Summary: Vicarious Surgical enhances spatial computing in healthcare with its advanced robotic system designed for minimally invasive surgery. Their technology integrates 3D visualization and precision robotics to interact within a small incision space, offering surgeons enhanced control and situational awareness. This approach minimizes patient trauma and improves surgical accuracy, representing a significant leap in integrating spatial computing with surgical procedures.
Head Office: Waltham, Ma
Funding: founded in 2014, $43.2M raised
Entertainment & Gaming
Description: Spatial computing transforms entertainment and gaming by creating immersive, interactive environments where physical and digital elements are seamlessly integrated. This technology enhances user engagement by allowing more intuitive and natural interactions within virtual or augmented realities.
Startup Example: Spatial.io
Summary: Spatial.io is an immersive platform that enables users to create and explore virtual environments for diverse applications such as gaming, education, and experiential marketing. Utilizing 3D modeling and virtual reality technologies, the platform supports user-generated content, allowing creators to build interactive experiences that can be accessed on web, mobile, and VR devices. Spatial.io is designed to foster a vibrant community of developers and users, promoting engagement through a multi-platform ecosystem.
Head Office: Cincinnati, OH
Funding: founded in 2016, $2.1M raised
Synthetic Environment and Spatial Simulation
Description: Synthetic Environment and Spatial Simulation involves creating digital replicas of real or imagined environments for training, analysis, or decision-making. These environments simulate real-world conditions and allow users to interact with them in a controlled and immersive way, making them essential for planning, analysis, and training in sectors like defense, urban planning, and gaming.
Examples:
Military Training Simulations: Virtual battlefields where soldiers practice strategies and tactics without real-world risks.
Urban Planning Simulations: Virtual models of cities for testing infrastructure changes, traffic patterns, and disaster response strategies.
Startup Example: Hadean
Summary: Hadean provides spatial computing technology to create synthetic environments for planning, training, and decision-making. They offer scalable supercomputing platforms that leverage AI to enable fast, autonomous decisions in defense, entertainment, and enterprise sectors. The technology supports large-scale simulations, enhancing real-time operational capabilities across various industries.
Head Office: London, England
Funding: founded in 2015, $70.6M raised
Watch More:
Startup Example: Blackshar.ai
Summary: Blackshark.ai specializes in creating highly detailed, AI-driven 3D digital twins of the planet, leveraging real-time satellite and aerial imagery for applications in geospatial analytics, simulation, and urban planning. Their products include Orca™ Huntr for object detection, SYNTH3D for synthetic 3D environments, and specialized solutions like Digital Airports and Synthetic Training Data. These technologies are used in diverse fields such as flight simulation, risk management, smart city development, and more, providing scalable, accurate digital representations and analytics of physical spaces.
Head Office: Graz, Austria
Funding: founded in 2020, $35M raised
Watch More:
The Future of Spatial Computing
Integration with AI and Machine Learning: Enhanced AI models will profoundly influence spatial computing by improving the accuracy and efficiency of spatial data analysis. This will lead to innovations in navigation systems, dynamic environmental monitoring, and personalized augmented reality (AR) applications, as well as more sophisticated robots that interact with their environments in intuitive ways.
Advanced IoT Integration: The proliferation of IoT devices will enrich spatial data inputs, significantly impacting urban planning, logistics, industrial, and infrastructure management. These devices will facilitate the seamless integration of real-time spatial data across networks, enhancing operational efficiencies and enabling smarter city solutions.
Enhanced Real-Time Data Processing: Future spatial computing will benefit from advancements in computing power and algorithm optimization, allowing for the real-time processing of complex spatial data. This capability is crucial for applications requiring immediate responses, such as autonomous vehicles and emergency response systems.
Ubiquitous AR/VR Including Mobile AR: AR and VR technologies will become common in daily activities and across various industries, incorporating headset-free XR to make digital interactions more accessible and immersive. This trend will see XR becoming more integrated into professional tools, educational programs, and recreational activities.
Improved Wearable Technology: Wearables will evolve to be lighter and more powerful, with enhanced resolution and longer battery life. This will make AR glasses and VR headsets more practical for continuous use in various settings, from industrial applications to consumer entertainment, further mainstreaming spatial computing technologies.
Reference
[1] CES 2024: What Is Spatial Computing? https://www.forbes.com/sites/cathyhackl/2024/01/06/what-is-spatial-computing/
[2] The Spatial Computing Landscape https://www.ngpcap.com/insights/the-spatial-computing-landscape
[3] Spatial Computing Market Size, Share, and Trends 2024 to 2034 https://www.precedenceresearch.com/spatial-computing-market#:~:text=The U.S. spatial computing market,share of 32%25 in 2023.
[4] Minds in 3D: AI is Seeing the Unseen with Spatial Intelligence https://rupali-patil.medium.com/minds-in-3d-ai-is-seeing-the-unseen-with-spatial-intelligence-12b0b1fbf2f3
[5] FutureMapping: The Computational Structure of Spatial AI Systems https://arxiv.org/pdf/1803.11288
[6] Beyond AR vs. VR: What is the Difference between AR vs. MR vs. VR vs. XR? https://www.interaction-design.org/literature/article/beyond-ar-vs-vr-what-is-the-difference-between-ar-vs-mr-vs-vr-vs-xr