Augmented reality (AR) technologies enable the projection of virtual content into the visual field of a viewer.
Hardware for AR can be realized at every stage of the visual pathway.
Technologically immature and socially unacceptable, implants external to the medical space are rare at present.
Optical solutions generally increase in complexity as form factor decreases and for systems closer to the eye.
Accordingly, transparent VR headsets and optical AR headsets have been subject of the most successful R&D
efforts at present and will likely provide the richest AR feature set in the near future. Smart glasses, a more
acceptable form factor, have garnered massive research funding but pose more substantial technical challenges.
Smart glasses are unlikely to have OSs or support applications substantially more complex than those of present
smart watches in the near future. Smart contact lens R&D has been isolated primarily to the startup Mojo
Vision, which continues to struggle with the most basic technical challenges of a MVP.
AR software, inheriting algorithms and environments from the more mature VR space, generally exceeds
hardware capabilities. Research continues in AR specific domains, particularly SLAM (+associated problems),
and generally in attempts to reduce computational requirements. As UI/UX of AR differs substantially from
that of planar display devices--interaction with software features occurs in 3D, rather than 2D space--existing
application ecosystems are rarely suitable for migration. Efforts to realize AR application ecosystems have
accelerated as of late. Strategies include modifications to existing VR infrastructure (Oculus/Facebook) or
Apple’s inclusion of LIDAR and associated AR development features in product offerings of late. Transparent
VR headsets and AR glasses are likely to be the first successful realizations of AR in consumer markets but
represent entirely distinct performance domains, enabling distinct software. AR glasses, designed for full day
use, will be correspondingly constrained in size and mass, limiting computational power and optical quality.
Applications in the glasses domain, if omnipresent, will likely conform to performance constraints similar to
those on present smartwatches. Applications in this domain will probably provide supplemental, contextual
information, rather than immersive experiences. A typical example is migration of data from maps--overlaid
directions or restaurant ratings, but the range of possible applications is vast. Who imagined Uber at the
conception of the iPhone? Transparent VR headsets, wherein cameras on the exterior of a VR headset enable an
AR experience, will likely capture enterprise and home use. A larger form factor enables greater onboard
computation and a higher quality of the optical experience, producing a more immersive experience. Because the
eyes are obscured by the headset, transparent VR is likely to be unacceptable in social situations or any context in
which device failure (obstructed vision) is a safety hazard. Applications will likely be similar (and perhaps
competitive to) those in the VR space: games, entertainment, and metaverse-style social interactions. On longer
timescales, transparent VR technologies might come to render surroundings so realistically as to be
imperceptible. Alternatively, AR glasses and headsets might improve in optical performance as to achieve the
same effect. In either eventuality, the dream of AR, amalgamation of real and virtual worlds, rendering virtual
goods and services substitutes for some real goods and services, will be possible. Creation of a metaverse is likely
independent of the eventual dominant hardware.
Though major players are developing their own chips for accelerated AR computations, any
innovation in graphical or neural computation, particularly reductions in power consumption and thermal
output, is relevant to AR (and the whole of the mobile space).
Recently, a common optical strategy has been a transition to Maxwellian systems, which map an image
to the retina with relative invariance to changes in the lens of the eye’s accommodation. Direct retinal scanning is
the most common of these techniques: the associated MEMs and laser systems might be locusts of innovation.
More exotic retinal scanning schemes, such as those eliminating floaters or providing variable depth of focus, are
also particularly valuable. Alternatives to Maxwellian systems, like LCOS or microLED displays, are worth
diligence. Innovations in optical waveguides, such as the emergence of higher-performance holographic
materials, improved-yield or improved-precision manufacturing techniques for holographic and diffractive
waveguides, are valuable to nearly all AR systems.
Interaction with software in 3D space is a subject of intense R&D. Hands and eye tracking hardware
are with BCIs the most likely input sources; any technology that allows their data collection at high resolution is
relevant to AR. See, for example, Facebook’s acquisition of CTRL-Labs, or Apple’s internal efforts at hand
positioning and input collection using smart rings.
Software: More uncertain, but firms producing software relevant to a metaverse, as realized in the near future by
low performance AR glasses worn by consumers for the duration of the day or higher-performance transparent
VR systems for shorter home use.
“Why is making good AR displays so hard?” provides a basic introduction to present AR hardware challenges.
Karl Guttag writes deeply technical reviews of consumer AR hardware as released and is probably the best
source to understand the short-term evolution of the optics space. Optical Architectures for Augmented-,
Virtual-, and Mixed-Reality Headsets (2020) is a detailed, current review of AR hardware. While not
mathematically rigorous, at times, the text assumes a deep understanding of the relevant optics. Holography:
Principles and Applications (2019) is an excellent introduction to holography relevant to AR (such as
Akonia/Apple’s volume holograms) and Introduction to Computer Holography (2020) to the relevant
computational methods. Handbook of Optics is a comprehensive optics reference that should answer any
remaining question.
AR UI design is detailed in 3D User Interfaces (2017). See Facebook, Microsoft, and Apple’s AR developer
resources for an understanding of present application possibilities.
.png){kind=icon}
