NVIDIA Content-Aware Fill Tool Can Reconstruct Photos And Faces

Adobe Photoshop is pretty good when it comes to filling in missing pixels of images. Its content-aware tool considers the surrounding pixels to make up for the lost ones and presents a final result that isn’t too far off from the original. Faces, however, have been harder to edit and restore this way. That is, until NVIDIA presented its content-aware artificial intelligence (AI).

Given a picture with erased features, NVIDIA’s AI is able to understand what the subject in an image should look like. Its neural networks are capable of identifying what’s missing from a face. If it’s an eye, it reconstructs one instead of filling the space with nonsensical pixels.

The tech developed can fill holes in photos as well as remove what shouldn’t be there to begin with. To reach this level of competency, the AI was trained on 55,116 random masks of holes and streaks superimposed on a plethora of photos. By examining the before and after pictures, the AI could construct the image once more.

“To the best of our knowledge, we are the first to demonstrate the efficacy of deep learning image inpainting models on irregularly shaped holes,” the researchers wrote in the paper “Image Inpainting for Irregular Holes UsingPartial Convolutions,” where the whole process is described in detail.

This announcement doesn’t come as a shock as machine learning enhancements have been on NVIDIA’s radar lately. The company made headlines more than once with their framework advancements: in December, by making it snow in California, a couple of months before by creating new faces, starting from celebrity pictures.