Beyond the Hype: Assessing Apple's Vision Pro and the Future of Spatial Computing
How long until we develop the tech to match our sci-fi dreams?
On June 5th, Apple announced the development of the world’s first spatial computer in their new Apple Vision Pro. And over the last month, it has created quite the buzz in the tech community, spanning reviews that love it, question it, or outright bash it. As always, though, Apple has managed to capture pop-culture’s and technologists’ imaginations as to whether Apple will be the global leader in two consecutive computing innovations: mobile computing, starting with the release of the iPhone in 2007, and spatial computing in 2024.
An exercise in definitions
As if the past few years haven’t already gifted us enough buzzwords and acronyms that we have no real grasp of in our day-to-day conversations: generative AI, AGI, LLMs, the metaverse, AR/MR/VR, etc., spatial computing has been added to the list. These tech terms are all increasingly intertwined, making it even harder to actually understand how each one contributes to the innovations of today/the future. So without boring you too much, here is what you need to know to understand why exactly the Vision Pro is (potentially) so impressive:
Virtual Reality (VR) - a completely simulated, digital experience that uses pose tracking and 3D near-eye displays to give the user an immersive feel of a virtual world. The physical or real-world environment is entirely blocked out.
Augmented Reality (AR) - a composite view of physical elements and digital elements, without interaction between digital and the physical world elements.
Mixed Reality (MR) - a composite experience of physical and digital elements where the user can see and interact with both the digital elements and physical elements of the environment (sometimes called Extended Reality/XR).
Spatial Computing (SC doesn’t have the same ring to it) - the integration of traditional computing with 3D location, haptic feedback, and eye tracking to develop user interfaces. It’s an umbrella term that incorporates each aforementioned “reality” to improve the way we interact with technology.
Apple claims that the Vision Pro is the world’s first spatial computer, meaning that it can seamlessly blend VR, AR, and MR into one device that adapts to the user’s context and preferences. Yet, their peers at Microsoft and Meta have already released spatial computers to the market- seemingly without anyone noticing. So is Apple actually ushering in a new era, or do we have a new Google Glass on our hands?
The Hardware
The Vision Pro is essentially a pair of sleek-yet-bulky glasses that can project high-quality holograms onto the user’s field of view, as well as track the user’s eye movements, gestures, and voice commands. As with other devices on the market, it can also sense the user’s surroundings and create realistic interactions between the digital and physical elements. But where Apple is banking most of their success is in the sheer amount of research and investment (over 5,000 patents were filed for Vision Pro alone) into creating their own hardware components and patents to integrate the Vision Pro into their fleet of existing products.
The Vision Pro is powered by Apple’s high-performing A15 chip, and has been optimized for spatial computing by adapting many of their features; such as SpatialKit, a framework for developing immersive applications; SpatialSiri, a voice assistant that can understand natural language and spatial context; and SpatialFaceTime, a video calling service that can create lifelike avatars of the participants. Additionally, they’ve adapted nearly all of their accessories to Vision Pro as well; such as the SpatialPod, a wireless controller that can simulate haptic feedback and provide precise input; the SpatialPen, a stylus that can draw and write in 3D space; and the SpatialEarPods, a pair of earbuds that can deliver spatial audio and noise cancellation.
Where I think Apple will (eventually) sway most of the public, though, is as simple as the difference between a blue and green bubble today: integrating Vision Pro into their product “ecosystem”. Apple will afford users to seamlessly transition tasks between MacBook and VisionPro, opening up opportunities to communicate, collaborate, and create beyond the bounds of a regular screen, keyboard, and mouse.
The company has also partnered with several content providers, such as Netflix, Spotify, Disney+, and Epic Games, to offer exclusive content for the Vision Pro users. For a more technical review of what all that truly means and why it’s so impressive, check out Marques Brownlee’s initial thoughts on Vision Pro.
Is Vision Pro the Future?
Has Apple invested enough R&D to create an inflection point in what devices we use and how we use them? Only time will tell. As alluded to earlier, there is a chance that it’s simply too great a change from our current habits and meets the same fate as Google Glass and other wearable technologies. Some critics argue that the Vision Pro is too expensive, too bulky, and too intrusive for mainstream adoption. Rather than easily slipping into our pockets, the Vision Pro has to be hauled around with an external battery and connection to an iPhone/Macbook, making it more like the original “Brick” mobile phone than the iPhone.
But it’s hard to ignore the obvious comparison of how the original iPhone was received by the public in 2007: it was too expensive, overhyped, and didn’t fully live up to its technical promise. Will any of my friends and I be using Vision Pro to hang out in the next 3 years? Not unless this new hobby makes me rich. Yet, Apple’s continuous ability to capture our imaginations reminds me of an old quote:
“A critical . . . reading of science fiction is essential training for anyone wishing to look more than ten years ahead.” - Arthur C. Clarke
So, let’s take off our cynical and dystopian lenses for just a moment to examine the fundamental change to digital interactions that is coming over the next 10 years.
This post was originally inspired by Megan N’s Notes and Tips on Spatial UI upon the release of the Vision Pro, and is certainly worth checking out in full. Most notable to non-designers though, are the following:
We are no longer bounded by screens (duh)
GIF via UX Collective While pretty obvious, I don’t think it can be overstated just how strange and freeing it will be to live in a digital world that pervades our physical environment. With spatial computing, we can enhance our perception of reality with digital information and feedback, seeing data and information overlaid on our environment in a much more visceral sense. We’ll also be able to feel haptic sensations that simulate touch or movement, improving our awareness, understanding, and decision-making across just about any context. If you’re the type of person to spend hours customizing your home/lock screen, prepare for creative bliss (and possibly overstimulation).
A Return to the Original “Window” - Our Field of View
(source) In a similar sense to the previous, the way information will be laid out to us is no longer constricted to screens. While we’ve all trained ourselves to scan webpages in the most efficient way possible, finding key points of info, patterns/insights in data, or whatever else you’re looking for will be much more intuitive to how we naturally interact with the world.
Our Eyes Are Back in the Driver’s Seat
Rather than using your cursor to navigate, scroll, hover, and select items on the web, our eyes will be able to interact with digital objects like they never have before. Simply focusing on a component visually will change it into a “focused” state, where the system will respond to our visual movements/processing in ways that are much more natural than on desktop or mobile devices. And while I haven’t experienced anything quite like this myself, this feels the most like the future to me.
Dipping My Toes Into Spatial Design
As I’ve mentioned, it’s difficult to know when exactly this way of digital life will enter the mainstream and pervade our lives to the same degree as mobile phones- it’s strange to even think that another device will replace the black mirrors we’ve become so accustomed to. As it stands, I’m optimistic on the timeline, and consider it nearly an inevitability. And while I’m no developer, I have been playing around with what the future might look like, and the unique considerations of building experiences in this new digital environment. So over the weekend I played around in Figma and created a file to explore what Spotify might look like in the Vision Pro environment:
Explore the file in full here. I’ll be continuing to iterate on this and explore other forms of Spatial UI and design- feedback, tips, and recommendations are welcome!
Thanks for getting all the way through my second post. Consider liking and subscribing below :)