Adobe Photoshop is pretty good when it comes to filling up missing pixels in images. Its content-aware tool considers the surrounding, present pixels to make up for the lost ones and some of the time, the final result doesn’t look half bad. If you don’t have to work on portraits, that is. Faces are much harder to edit and restore in an editing program unless… you’re called NVIDIA and AI at your disposal.
Machine learning enhancements have been on NVIDIA’s rada a lot, lately. The company made headlines more than once with their framework advancements; in December, by making it snow in California, a couple of months before by creating new faces, starting from celebrity pictures.
Now, their AI is able to understand what the subject in an image should look like. Its neural networks are capable of identifying what’s missing from a face. If it’s an eye, they reconstruct one instead of filling the space with nonsensical pixels.
The tech developed can fill holes in photos as well as remove what shouldn’t be there to begin with. To reach this level, the Ai was trained on 55,116 random masks of holes and streaks superimposed on many, many photos. By examining the before and after pictures, the AI could construct the image once more.
“To the best of our knowledge, we are the first to demonstrate the efficacy of deep learning image inpainting models on irregularly shaped holes,” the researchers wrote in the paper“Image Inpainting for Irregular Holes UsingPartial Convolutions”, where the whole process is described in detail.