NVIDIA AI Made It Snow In California To Train Self-Driving Cars

In the Golden State, where it (almost) never rains, self-driving vehicles have a splendid time learning the roads and the rules of navigation. But that’s not always a good thing; what will they do in more moody locations? Where winter is visible in snowfalls? That’s why Nvidia made it snow in California – literally. #softwaremagic

Nvidia created an AI framework capable of envisioning the same sunny street in dramatically different weather conditions – during rain showers or snowfalls – and even at night. The technology employed was a generative adversarial network or GAN. A GAN is based on the work of two separate neural networks. One makes the data, the other judges it, rejecting the ones that look “off”. In time, the AI becomes better and better at giving results.

Where Nvidia innovated is at the labelled datasets level. Most GANs need that to generate data, meaning it needs pictures taken during the day and during the night at the same location to transform the scene. The company managed to skip that part though, freeing up engineers’ time significantly. “We can use our method to translate sunny California driving sequences to rainy ones to train our self-driving cars”, stated Nvidia.

Weather is not the only thing the AI can play with. It can also transform one breed of pet into another and change persons’ facial expressions. In that respect, you’ll find it does similar work to the one seen from mobile applications like FaceApp. The extent to which the algorithm can go is pretty daunting, if you ask us. Check out the full research paper here.