Google Has A Tech That Makes VR Headsets Invisible

Virtual reality is often called an “immersive experience” and indeed, it is – for the person with the headset on. The rest of us are left to watch him or her with a big, chunky headset on, smiling and moving around in the VR environment. This could change though, with initiatives like Google’s new technology to make the headset almost invisible #realitymagic

Google Machine Perception researchers, Daydream Labs and YouTube Spaces have worked on a solution to make social interactions easier and comfortable in virtual reality. Researchers used a mix of machine learning, computer vision and advanced rendering tools to reveal the “identity, focus and full face in mixed reality”. It was time consuming but not so difficult to accomplish as you might believe.

The team made a dynamic 3D model of the user’s face first and tried to capture the most common facial expressions. This helped in simulating blinks and looks, even when the user is wearing an opaque headset.  Then, they used an HTC Vive headset that included eye-tracking to capture the gaze of the user from under the VR gear.

All that was left to do was to align and blend the 3D model with the camera’s video stream. Then, the Google team composited the figure in the VR realm: “The end result is a complete view of both the virtual world and the person in it, including their entire face and where they’re looking.”

Ta-da! Granted, that headset still looks like a scuba mask, but at least you can see the guy’s facial expressions, where he is looking and how everything fits together. This is a key technology for VR applications that allow multiple gamers or that encourage virtual conferences.