Google’s DeepMind AI Can Lip-Read Better Than A Human

Google’s DeepMind artificial intelligence system is building up its resume by adding skills on a daily basis! After several successes, it recently outperformed a human professional at lip reading #machinemagic

The University of Oxford and Google’s DeepMind AI was trained to lip-read using 5000 hours from six TV programs, broadcasted in the period January 2010 – December 2015, with a total of 118.000 sentences. Then, the AI was tested using data from the same programs broadcasted during March 2016 – September 2016. Google’s DeepMind successfully deciphered 46.8% of all words while a human professional was only able to recognize 12.4% of 200 randomly chosen words without error.

What happened? In the training period, by associating sounds with mimic, a computer system was able to figure out how many times they were out of sync and realign them. Afterwards, the 5000 hours of data (video and audio) were processed automatically and eventually, entire sentences like “We know there will be hundreds of journalists here as well” were deciphered correctly.

The BBC data set will be released as a training resource. Who knows, maybe soon we’ll be able to give commands silently to our mobile assistants, far from eavesdroppers!