Roboticists at Queensland University of Technology are using neural networks to help robots quickly and accurately grasp objects in cluttered and changing environments.
“We have been able to program robots, in very controlled environments, to pick up very specific items. However, one of the key shortcomings of current robotic grasping systems is the inability to quickly adapt to change, such as when an object gets moved,” said Dr Jürgen Leitner. “The world is not predictable – things change and move and get mixed up and, often, that happens without warning – so robots need to be able to adapt and work in very unstructured environments if we want them to be effective.”
The chosen approach is a a real-time, object-independent grasp synthesis method for closed-loop grasping called a ‘generative grasping convolutional neural network’, which predicts the quality and pose of a two-fingered grasp at every pixel in the visual field.
By mapping what is in front of the robot using a depth image in a single pass, the robot doesn’t need to sample many different possible grasps before making a decision, avoiding long computing times, according to the University.
In tests, an 83% grasp success rate was achieved on a set of previously unseen objects with adversarial geometry, as well as 88% on a set of household objects that were moved during the grasp attempt, and 81% when grasping in dynamic clutter.
In 2017, the team behind this research won the Amazon Picking Challenge with a robot called CartMan which looked into a bin of objects, decided where the best place was to grasp an object, then blindly went to pick it up.
With the new method, the robot processes images of the objects within its view in ~20ms – fast enough to allow it to update its decision on where to grasp an object.
“This line of research enables us to use robotic systems not just in structured settings where the whole factory is built based on robotic capabilities. It also allows us to grasp objects in unstructured environments, where things are not perfectly planned and ordered, and robots are required to adapt to change,” said Leitner. “This has benefits for industry – from warehouses for online shopping and sorting, through to fruit picking. It could also be applied in the home, as more intelligent robots are developed to not just vacuum or mop a floor, but also to pick items up and put them away.”
This work is being presented at the ‘Robotics: Science and Systems’ conference at Carnegie Mellon University this week, in a opaper called ‘Closing the loop for robotic grasping: A real-time, generative grasp synthesis approach’.