MIT is using radar and artificial intelligence to sense people’s postures and movement, even through a wall.
Led by Professor Dina Katabi from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL). the project, ‘RF-Pose’ can mimic a real person as they move, creating a dynamic stick figure that walks, stops, sits and moves its limbs.
One challenge the researchers had to address, according to MIT, is that most neural networks are trained using data labelled by hand. A neural network trained to identify cats, for example, requires humans to go through a big data-set of images labelling each ‘cat’ or ‘not cat’, but radio signals can’t be easily labelled by humans.
The solution was to collect images with a camera at the same time as the radar collected data from people of people walking, talking, sitting, opening doors and waiting, then extract stick-figure images from the photos which were presented to the neural network alongside the corresponding radio signals.
This means that the network was never explicitly trained on data from the other side of a wall which, according to the university, “made it particularly surprising to the MIT team that the network could generalise its knowledge to be able to handle through-wall movement. “If you think of the computer vision system as the teacher, this is a truly fascinating example of the student outperforming the teacher,” said researcher Professor Antonio Torralba.
Besides sensing movement, the system can apparently identify somebody 83% of the time from a line-up of 100 individuals.
So far, the model outputs a 2-D stick figure, but the team is working to create 3-D representations that include small movements, such as the trembling of a hand.
Potential application are sought in healthcare: The team said that the system could be used to monitor diseases like Parkinson’s and multiple sclerosis, providing a better understanding of disease progression and allowing doctors to adjust medications accordingly. It could also help elderly people live more independently, while providing the added security of monitoring for falls, injuries and changes in activity patterns.
Besides health-care, other possible uses are video games where players move around the house, or in search-and-rescue to locate survivors.
The work will be presented later this month at the Conference on Computer Vision and Pattern Recognition (CVPR) in Salt Lake City.