3D Portrait, LG’s Secret Selfie Weapon, Lets You Create Realistic Shadows

Last year, Simon Fitzpatrick, Senior Director of Product Management at Xperi, predicted AI and 3D will continue to shape mobile photography, as users will go from enhancing different features to customizing their looks. This year, Simon revealed that FotoNation, part of Xperi, played a major role in offering users the means to recompose their portraits. The company teamed up with LG to bring a unique feature on the LG G8 ThinQ: 3D Portrait Relighting.

We met Simon Fitzpatrick at Mobile World Congress, after taking a look at LG’s new offering, the G8 ThinQ. Turns out, the Spotlight effect we had noticed on the G8 ThinQ was done with the help of FotoNation (Xperi): “LG branded it Spotlight, we call it 3D Portrait Relighting, one element of our 3D Portrait solution.”

“It’s like augmented reality with lighting”

Our first impression of the camera effect was that it thinks and acts as a team of photographers preparing for a studio portrait shoot. In a blink of an eye, the software is able to understand the existent illumination and can recommend different lighting setups to get the portrait you desire, but might not have the technical background or equipment to achieve. It does so in seconds, whereas professionals would need more than a couple of minutes.

That’s not all, though. Impressively, 3D Portrait allows you to create virtual light and cast shadows out of thin air! “It’s like augmented reality with lighting, which is really unique on the LG phone. We’re the first to do it”, explained Simon.

This magic continues after you press the shutter button, too, because the information captured with the 3D sensor remains: “You can tweak it or try to get it right the first time. We’re trying to enable both experiences, but do it all in a small form factor, where there are no fancy keyboards, make it as simple and accessible to everybody.”

“Our vision is to
give people the ability to do amazing portrait photography with the touch of a button”

All of those times we wished we’d have access to a studio and equipment to post professionally-looking portraits on social media? They could be part of the past with this mobile technology. FotoNation definitely envisions that future: “Our vision is to give people the ability to do amazing portrait photography with the touch of a button whereas a photographer would use a deflector, a flash unit, fancy cameras and lenses.”

Right now? Their 3D solution is helping build that future by giving OEMs the chance to spoil their consumers with a suite of tools to relight a portrait, add lens flare or bokeh and enhance their natural features, in a realistic, not-doll-like way.

“We’re going for realism with skin texture – still needs to be there [instead of] blurring it as some do -, spot removal without removing a beauty spot. [We don’t want] to take 200 years off of someone who is 50”, Fitzpatrick explained.

In the next couple of years? They want to push imaging enhancement further, by training algorithms to optimize power performance, while ensuring high frame rates for videographers. That said, Fitzpatrick knows hardware providers have to pitch in as well; 3D sensors’ resolution needs to go up first, without draining the battery too much.

It’s quite the balancing act, having OEMs, hardware and software providers on the same page and in the same time. We can’t wait to see what mobile teams will take the lead first!