CSAIL System Teaches Robots To Handle Objects They Have Not Seen Before

Robots nowadays are pretty smart, especially when compared to their predecessors from a few years ago but they still have a long way to go – most robots currently in use in factories and on assembly lines are pre-programmed to handle the objects they will be manipulating, with no room left for anything else. If the machines face objects they don’t recognize, they usually just end up breaking them by handling them in the wrong way.

To combat this issue, MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) got to work and developed a system called kPAM (KeyPoint Affordances for Category-Level Robotic Manipulation), which teaches the robots how to pick up and manipulate objects they never saw before.

So, how does it work? In layman’s terms, the system creates visual roadmaps of unknown objects by allowing the robot to perceive them as 3D keypoints thus allowing the machine to handle them easily and accurately.

Most of the algorithms factory robots will estimate all their data as opposed to points, which works for one single object but results in misuse of the objects that are shaped in a different way than what they’re used to, especially for ‘odd’ shapes such as cups or shoes. The kPAM on the other hand can accommodate that variation thanks to the 3D keypoint system.

“Understanding just a little bit more about the object — the location of a few keypoints — is enough to enable a wide range of useful manipulation tasks.” Russ Tedrake, MIT professor and senior author of the study said “And this particular representation works magically well with today’s state-of-the-art machine learning perception and planning algorithms.”

kPAM only needs three coordinates for shapes such as cups, for example (bottom, center and top center), while more complex shapes like shoes will only require around six of them.

“Whenever you see a robot video on YouTube, you should watch carefully for what the robot is not doing,” Tedrake said “Robots can pick almost anything up, but if it’s an object they haven’t seen before, they can’t actually put it down in any meaningful way.”

For the time being, the system is still undergoing tests and the engineers are collecting data on how they can further improve kPAM. However, hey do hope that, in the future, the system will be complex enough to bring considerable improvements to factory robots as well as allow the machines to accomplish tasks such as wiping down kitchens or unloading dishwashers,