What is Digital MR?

I noticed many news sites speculating wildly about VRVANA and the Totem this week. Business Insider says:

'...the Vrvana Totem is said to be able to overlay full, colorized digital images over the real-world. That would be a step above rivals like the Microsoft HoloLens, which only project transparent "ghost" images within a limited field-of-view. If the Vrvana Totem lives up to those lofty promises, it could give Apple a major leg up in augmented reality, which the company sees as the next major computing platform.'

The fact that the Totem uses video cameras to show the 'real world' has been totally missed which is a key difference from HoloLens, Magic Leap, Daqri or ODG devices. In reality, the device is more like AR on a tablet but in stereo than devices that mix optical and digital views together. The problem is most people reporting on these devices just do not understand enough about how they work or have even tried them to appreciate what they can and can't do well.

CEO of VRVANA Bertrand Nepveu gave me a personal demonstration of the Totem at Siggraph in 2016 and I was quite impressed.

Any MR device that uses digital cameras to show the outside world also has to deal with adding latency to the real world which optical 'pass through' devices don't have to contend with. This latency or lag can easily stop the experience from feeling 'real'. The latency on the Totem was not too bad and I found flying a virtual helicopter around the show floor and landing it on the stand quite engaging.

Digital MR devices do have their advantages though over their optical cousins. Here are some of the key ways:

• The field of view (FOV) of the visible world and digital world always match as they are both being displayed on the same screen.
• The FOV can be much larger than current Optical MR devices.
• The visual quality/feel of both the real and digital world are the same as they are both being displayed on the same screen. This creates a more seamless blend of both worlds. With perfect registration and alignment of elements across the view.
• Occlusion is much easier to do allowing virtual objects to go in front or behind real world objects completely seamlessly. Digital objects can more easily appear to interact with the real world.
• VR and Optical MR devices still don't have stereo cameras (as its actually quite hard to match position across optical/video/tracking sensors) so you can't record your stereo view for you or someone else to review. Digital MR devices can do this very easily. Note this can work best on 2D screen with 3D glasses rather than an HMD as the view will not have independent control of head position.
• With Digital MR, the fact that the real word is also digital, means light can be subtracted from it, which is something optical MR devices cannot do. What this means is that if you want your virtual object to cast a shadow on the real world or cover up a source of light like a window it can do quite easily. Devices like the HoloLens interpret black virtual objects as transparent as they can't take away light from your view. Some describe light subtraction as 'Diminished Reality' and it will soon become an important 'next step' for immersive MR.

In time, it is likely that VR and MR will merge as VR headsets start to incorporate stereo cameras and AR/MR headsets start using some kind of LCD technology to also allow the blocking-out of the real world. The latter is harder than it sounds right now due to registration/alignment difficulties between different display systems.

We are already starting to see devices like the ZED Mini that help you adapt your VR Headset for this.

As Monika Bielskyte of All Future Everything says...

The digital world will soon enough be enmeshed with the physical world in such a way that our ‘reality’ will be the transparency mode that we choose.

Is 'Windows Mixed Reality' Digital MR?

The new 'Windows Mixed Reality' headsets from five Microsoft partners have 2 digital cameras on the front and are called mixed reality devices yet don't currently use the cameras for pass-through OR have optical pass-through.

At the moment all the headsets only use their cameras for SLAM tracking or warning you of physical boundaries not creating a blended view of real and virtual worlds. They also use the Windows Holographic platform originally created for the HoloLens to help develop a common platform across a range of devices for working with 3D objects for VR/MR.

Other actual Digital MR Devices

• Canon MD-10 MREAL display
• Sulon Cortex
• Intel's Project Alloy - Discontinued.

There are quite a few others out there, some old some new. Keep a look out for them.

Conclusion

Whether it is VR, AR, Optical MR or Digital MR, the important consideration is NOT which technology is better but which best meets your specific needs. You can only compare their abilities for a particular job, not in general against each other, which is something all tech reporters seem to like doing. So look at the visualisation problem you have and then find the device that best solves that problem.