An alien spaceship has blasted a hole in my ceiling and is landing on the floor in front of my bookcases.
It’s a game on the Meta Quest 3 headset named “First Encounters.” My office is fully visible through the passthrough cameras. In the picture above, you can see a giant moon hanging in the sky outside. Alien puffballs are about to break through the walls and bounce onto my chair, hop onto the credenza, hide under the desk, and generally be a nuisance. I’m ready, dammit. I shoot them with a raygun to capture them in a Ghostbusters-style container, which launches away through the hole in the ceiling at the end of the game. (First Encounters is free and included with every Quest 3 headset.)
Meta and Apple are starting to provide mixed reality experiences. Today they require wearing the equivalent of a swimming mask – the Meta Quest 3 or Apple Vision Pro. In “passthrough” mode, it looks like you’re seeing your living room through clear glass.
The mixed reality experiences show digital objects that interact with the real world. In Starship Home, an alien plant sits on my desk, a giant spaceship viewport hangs on my closet door, and I can walk up and stare closely at the terrarium with the alien plants on the bookshelf. I am fully aware that I sound like I’m five years old but shut up.
Let’s take a minute to talk about terminology. This is the dialog going on in the industry today about these new experiences.
“This new technology ought to have a name. A mixture of the real world and alien puffballs. Let’s call it ‘mixed reality.’”
“Ooh, wait! It makes the real world even better by adding digital elements, there must be a more obscure word . . . I know, we can call it ‘augmented reality.’”
“No no no no no, I have one, this is too good, let’s say it’s ‘extended reality.’”
“Good efforts but there’s still a chance people might know what the hell we’re talking about, so let’s also use an acronym that doesn’t quite line up with any of them, like ‘XR.’”
“There’s no reason to be shy. Let’s use them all interchangeably and occasionally put up explanations of the differences between them that explain nothing because they’re all the same.”
Everyone agrees to use multiple terms to refer to the same thing. Then they have lunch with the team from Microsoft that uses the same word “Outlook” to refer to dozens of wildly different products. Two roads to the same result: maximal customer confusion.
Meta lowered the price of the Quest 3 to $499 a few weeks ago. It’s a great deal. Meta also introduced a lower priced model, the Quest 3S at $299, which is surprisingly high quality – there’s never been anything like it anywhere near that price point. If you’re curious and you can afford the Quest 3, treat yourself to it (better optics – things look better in subtle but meaningful ways), but don’t hesitate to get the Quest 3S if you’re on a strict budget or buying a gift for a pre-teen.
The Quest 3 has been available for a year. Meta delivered a firmware upgrade a couple of months ago that dramatically improved the passthrough – it’s clearer and the colors are better than when the headset first shipped.
First Encounters and Starship Home are early examples of how mixed reality might be used in games. Popular workout app Supernatural added a mixed reality mode a few months ago so you can see your walls and furniture as you whack floating objects in time to music. Tripp, a meditation and wellness app, lets you turn on passthrough so you can place relaxing objects around your room.
There are more mixed reality apps coming soon. Wall Town Wonders turns your walls into a tiny city filled with tiny people. Living Room brings a wildlife sanctuary into your living room.
What’s coming, though, is more than just a few new games. In hindsight we may decide that 2024 marks a turning point for mixed reality.
At the high end, Apple Vision Pro is almost exclusively for mixed reality, offering a crystal clear passthrough view of the world around you with screens hanging in space to watch movies or get work done. Vision Pro is not a consumer product, it’s more of a proof of concept for developers, but it appears Apple is committed to the platform for now and rumors are circulating that a cheaper version may arrive next year.
Meta surprised industry observers in April when it opened up the Quest operating system to third party developers, the exact opposite approach from Apple’s locked down operating systems. Meta is hoping to kickstart development of mixed reality apps even if that means competitors can release their own headsets to compete with Quest. (Meta’s has such a lead in hardware research and development that it will be hard for anyone else to be more than a runner up.)
So Meta is creating an open ecosystem and seeding the market with low cost, high quality devices. Meta has partnered with Lenovo, Asus, and Xbox for new devices, and the increased competition will spur hardware advances as well as encouraging developers to create applications and games. Consumers are becoming familiar with VR technology and the cheaper Quest 3S will help more people decide to experiment.
There’s a tipping point in this process where mixed reality turns into a mainstream product. It might be visible in the distance. I think I can see it through the holes in my bookcase. I’ll keep an eye out after I deal with the pesky alien puffballs.