Mapping the World for Augmented Reality

Augmented Reality (AR) is a key long-term focus for many of the largest tech companies in the world (i.e., Apple, Meta, Samsung, Qualcomm). As we have highlighted in past newsletters, billions of dollars in R&D spending is flowing into extended reality (XR) since many companies are viewing AR and VR as the next medium of consumption and interaction.

A lot of focus is being placed on building AR-first applications that are uniquely suited for glasses. However, we believe there is a core foundation that is missing before these applications can reach their true potential: an Augmented Reality Mapping Layer (ARML). This is a system that can hyper accurately determine the position and orientation of AR devices globally.

AR by definition is the process of enhancing your physical reality. This is far different from VR, which is designed to transport users to a separate and fully digital dimension. In this newsletter, we want to highlight and explain why we believe this mapping layer will be one of the most important developments in AR.

Scalable Content: As of today, the majority of AR applications are built to supplement natural vision. Examples include a tabletop role-playing game where digital scenes and characters come to life, or trying on virtual renditions of clothing. One of the most popular existing examples is Pokémon GO.

We believe AR content will be stored within this foundational AR Mapping Layer and give users the ability to access streamed content as needed. There will be no limits to the amount of content anchored to a location or experience and users will be able to choose what specific content they want to access.

Because AR devices need to be lightweight, portable, and comfortable to wear for hours on end, their hardware and storage capabilities will be severely limited. Small applications will be downloaded onto these thin client AR devices and content will be streamed in near-real time.

Within every location, event, or experience, content can scale infinitely based on the needs of the end user and developers because the digital world is not limited by the same confines as the physical world.

The New Social Network: When looking at why people flocked to social networks, the most obvious theme was to stay connected and share special moments in a one-to-many fashion. AR, supported by an AR Mapping Layer, has the potential to elevate social networks and interactions by better contextualizing updates within the real world and making the experience of those updates more real.

Events: Events like concerts or sporting events are valuable to individuals because they are one-time moments that can not be duplicated in a live setting. In the future, with the unlock that an AR mapping layer provides, individuals could go to venues or stadiums and re-experience events as if they were there.

Guided Tours: Museums, national parks, or even cities could leverage an AR mapping layer to create experiences that visually enhance tours or in-person experiences. Some cities, like Stirling, Scotland (BrigNews) have already built a city-wide augmented reality experience that allows for tourists to guide themselves through the city and learn about the history and culture.

Location-Based Education: Educational experiences can leverage specific data (like PictureThis for identifying plants in AR) about a location to help students or organizations better understand the area around them. Whether for geological features, plantlife, or wildlife, an AR mapping layer would help direct end users to specific locations to find and learn more about the environment around them.

Who Builds It and How?

The overarching question is how does someone, even a large tech company, build a project like an AR Mapping Layer of this scale? In April 2013 (the latest data point), it was estimated that Google had spent $1-2b for Street View “sending cars to take photos all over the world” (Business Insider). As of 2022, Street View has added more than 220 billion Street View images from over 100 countries.

To put into perspective how big of an endeavor an Augmented Reality Map Layer would be, Street View is still an ongoing project that started over 15 years ago with billions of dollars of investment. That being said, this is not how we believe it will be created for AR. Instead of a manual, centralized project, it will be community built. The model is more similar to Waze: cheap data collection through a popular consumer application. For reference, Waze obtains navigation information for free from a community of 40-50m volunteer users.

The initial data collection pipeline will most likely be created by a widely adopted consumer social application. In our view, the most likely creator and aggregator of data is Niantic. Niantic, from a content perspective, is one of the only companies that has truly cracked mass adoption in AR, specifically with Pokémon GO. It created a phenomenon where 100+ million people were exploring different locations (inside and outside) to find and capture different Pokémon.

This level of consumer adoption paired with Niantic’s Lightship SDK is unique and powerful. Lightship has a product called the Lightship Virtual Positioning System (VPS, see below), a solution that is being actively built by developers, surveyors, and players to collect millions of real-world location scans. As you can see from the map below, it is still very far away from being a complete scan of the real world.

As more consumer apps are created, more potential users will be able to collect data without the need to send volunteers or surveyors to specific locations (which is time and cost prohibitive). To date, only Pokémon Go (released seven years ago) has had widespread success as a location based VR application; many more popular location enabled AR games will be needed to build out VPS.

‍Takeaway: Augmented Reality is missing a core piece of infrastructure when it comes to positioning and orienting user devices in the real world. We believe that this AR Mapping Layer (ARML) is necessary to enable immersive experiences and sync content across users in real time. While there are applications that can be built without this infrastructure, we believe an AR mapping layer is needed for AR to reach its full potential and mass consumer adoption.