What’s next for augmented and virtual reality? (Part 1)

Now we are finally at a point where VR and AR are usable and capable of reaching a wider audience. I think it’s here to stay, despite the fact that the technology is still in its infancy and will develop in leaps and bounds over the next few years. Here are a few educated guesses about where things are headed in the next 2-5 years.

Small is beautiful

In every field of modern technology, we strive to make things smaller. Augmented and virtual reality devices are no exception. In this realm, size matters even more than in most other areas – mobile phones are a very handy size, but no amount of marketing videos will convince anyone that they’d be comfortable wearing Microsoft’s Hololens for a full workday.

There are two trends here: the high-end devices will still be clumsy for a few years. At the same time, more and more VR/AR applications will hit the low-end tech, a.k.a. high-end mobile devices. The clumsier and bigger devices will still have a role for a while though - especially in the industrial, business and gaming domains. Consumers, however, will not buy them until the devices, in essence, just disappear.

Less power

The word power can be understood in two ways: the raw computational power of the device or its energy consumption and battery life. The two are interlinked.

The capability of the computer running AR/VR applications not only has an impact on the demand for energy, but also on the size of the machine. Algorithms and the amount of calculation needed are still a big issue. Regardless of fancy sales videos, most applications still resort to rather crude graphics in order to save juice for the essentials, like a high frame rate. This means that even an affordable VR headset still requires a very powerful gaming engine that may cost several times the price of the actual device if you want to access to the most high-end experiences. Luckily, the prices are coming down as some VR-ready devices inch closer to the 1000 euro range.

Mainstream solutions need to run in existing consumer devices. Heavyweight VR can’t go mainstream if you need to buy too many expensive devices. The average consumer won’t buy a specialized VR headset, let alone an additional computer just to run VR. This limits the target market in heavyweight VR to gamers. One promising niche is the gaming consoles, where PS4 and Playstation VR have lead the way.

In addition to pushing more power into a smaller chip, manufacturers are working hard in an attempt to reduce computational complexity. Oculus’ announcement last year of a new algorithm called Asynchronous Space Warp is one example. It means they only need to calculate frames at half the speed and can deduce the missing frames from that information. The solution demands considerably less power, allowing for a less expensive (and smaller) GPU and computer to power it up. However, as Oculus is quick to note, this is not an optimal solution and the best experience still lies in the 90Hz domain. Similar technology called Asynchronous Reprojection is now available now for HTC Vive.

Battery technology has been a big concern in the development of almost all new technology. It impacts the range of automated cars, storing renewable energy or just having your fancy mobile survive a single work day. Batteries are developing at much slower pace than everything else, so this will continue to be an issue. Portable AR requires power. Any method for reducing the need is welcome and adds to its usability.

Understanding space

Mapping space in the real world is one of the most critical features in augmented/mixed reality. Microsoft Hololens is the first high-end device to do this with any degree of accuracy. Even standard physic demos are pretty impressive: things fall and roll in a fairly credible way, and walls open and crack in just the right places when the aliens are coming to get you.

Space mapping will trickle to full VR, too, as the boundaries between the two start dissolving. We’ll start using real-world props to augment the VR space to add to the immersion. Mixed reality as a term encompasses a wide variety of mixing the real and the virtual though the terminology is still somewhat vague.

Some devices like Facebook’s Santa Cruz portable VR prototype already include cameras to map the user’s surroundings. We are already using the room scale in high-end VR experiences - meaning that your movement in the (constrained) VR space is mapped one-to-one to your virtual world. This will be the default mode of operation for quite some time. It’s currently essential for limiting nausea and better linking the experience to the sensory experiences (balance, speed, position).

When we eventually get rid of physical limits like cords in VR, we’ll need a way to verify the user is not bumping into walls, and can walk around in the real world, meaning we’ll start including real world objects like chairs and generally mixing the real world into VR. For full immersion, the physical props mapped in VR might be needed long before VR headsets are standalone, freed from the server they now are connected to.

Augmented reality’s mainstream consumer applications will first appear on mobile devices. We’ll want more information in a store or a museum. We’ll use a lot of AR for entertainment. People don’t think twice of creating an anime or reindeer head with Meitu or Snapchat, and translating with Google Translate’s image replacement algorithm is becoming more commonplace. Visualising is a big deal - check out my “tattoo” done in InkHunter!

Look Mom what I did! (Just kidding.)

With current applications, we are mostly overlaying information on the screen instead of adding it to the space around us. Soon enough, people will want proper spatial mapping – we want those Pokemons really jump around you and react and bump to objects in real world, not just be overlayed on a video image.

To spped this development along, all high-end mobile phones will be space-conscious within a few years. Google's Tango has been showing the way in Google domain and Apple has already followed with two-cam iPhone7. They have yet to announce applications to use those cameras for spatial mapping, but the capability is now there.

In the second part - as we explore whether the technology is finally mature enough to have a mainstream impact - we'll look at the issues on a more detailed level: portability, control and touch.