Improving Human-Robot Interactions with Dynamically Complex Objects

Professor Dagmar Sternad, Biology, Electrical and Computer Engineering and Physics, recently received a $700,000 National Science Foundation grant title, “Learning to Control Dynamically Complex Objects” to improve human-robot interaction by exploring how humans manipulate complex objects and tools. Insights gained from the three-year grant, awarded to both Sternad and her collaborator, Professor Neville Hogan, Mechanical Engineering at MIT, may have significant implications for robot-assisted rehabilitation of individuals with impaired motor skills.

Achieving a deeper understanding of how humans control voluntary movement is one of the key objectives of Sternad’s research – an area she describes as “a big black hole in scientific knowledge” – and an important step in ensuring safe cooperation between humans and robots.

“Humans are still vastly better in controlling their movements than current robots,” explains Sternad, a fact she describes as “surprising” given the human system’s complexity, slow speed of information flow and internal variability. The question, according to Sternad, is “how do humans manage this seemingly over-complicated and poorly engineered system?”

The new line of research will test the hypothesis that control of the complex motor system is facilitated by building blocks – also referred to as “dynamic primitives.” Sternad and her team will seek to identify the kinds of “building blocks” or subsystems used by our motor control system.

Solving the puzzle of human movement

Sternad uses the example of walking on flat ground to illustrate the concept of dynamic primitives. “When we walk, we tend to set up a rhythm that is stable and operates without our paying much attention,” she explains. “Even if we step on a pebble, it’s not a problem and does not require explicit attention. We absorb it unless it is a big obstacle like a stone. Employing such stable subsystems is what we think happens for many other activities.”

To demonstrate more explicitly how dynamic primitives facilitate coordination, Sternad and her team will study how student volunteers learn to crack a whip – an exercise she chose for its challenging nature and “because sometimes you just want to have fun in the lab.” In Hogan’s lab, the team will examine how robots and humans interact by having them work side-by-side to spread a tablecloth over a table. “We control the robot similarly to how we think humans move using these dynamic primitives,” Sternad explains.

In these experiments, a human subject interacts with a flexible object in complex ways. “Whether it’s spreading a tablecloth, cracking a whip or carrying a full shopping bag, each task is a testbed or placeholder for a more general problem,” says Sternad.

Interest in solving the puzzle of human movement recently gained momentum with the focus on robotics. Even so, human movement remains “understudied and far from understood,” she concludes. “My deep conviction is that to understand humans and how they interact with their environment, you first have to find out how we control our movements.”

Abstract Source: NSF

Human sensorimotor capabilities vastly out-perform those of modern robots. Prior research suggests that humans achieve skillful movement by exploiting combinations of “dynamic primitives”, which are robust building blocks of coordination that simplify control. Interaction with complex objects – such as spreading a tablecloth – requires prediction which, in turn, requires mental representations of the objects and environment. This project explores the extent to which such mental models may also take the form of dynamic primitives. The project team will perform fundamental research exploring: how humans learn to manipulate a complex flexible object through the composition of dynamic primitives; the impact of explicit instruction on the acquisition of the mental models; and whether a primitive-based control structure to be implemented in a robot can achieve skillful manipulation of complex objects and fluid interactions between humans and robot. This project serves the national interest because the resulting understanding of human sensorimotor control and robot control methods may result in improved efficacy of robot-assisted physical rehabilitation after neuromotor injuries such as stroke, where safe physical cooperation with humans is fundamental and mutual learning is of paramount importance. The project will involve an educational component that provides engineering and research methods training to graduate and undergraduate students, as well as STEM outreach to individuals of all ages at the Museum of Science in Boston. Additional efforts will be made to attract and retain women into careers in science and engineering.

This research investigates a biomimetic approach to robot control that promise improved human-robot physical interaction during co-manipulation of flexible objects with complex continuum dynamics. Methods include skill acquisition experiments involving expert and novice human subjects to test how mental representations are formed and the extent to which they may be shaped by explicit instruction during training; a theoretical study of how motor learning may be facilitated using mental models based on dynamic primitives vs. lumped mechanical properties; and a study of human-robot co-manipulation of flexible objects wherein dynamic primitives provide the basis not only for robotic control, but also for communication and mutual learning between the human and robot partners. This research promises new insights into the manipulation of complex objects and tools, an area where humans still outperform machines.

This award reflects NSF’s statutory mission and has been deemed worthy of support through evaluation using the Foundation’s intellectual merit and broader impacts review criteria.

Related Faculty: Dagmar Sternad

Related Departments:Electrical & Computer Engineering