Creating a Human Avatar in Remote Locations

Computer engineering graduate student Rui Luo operates the robot that will be entered into the September 2021 semifinals of a $10 million global robotics competition.

A group of Northeastern University students led by MIE Assistant Professor Peter Whitney, has made it to the semifinals of the global Avatar XPrize competition which aims to create an avatar system that can transport human presence to a remote location in real-time.


The robotics team advances to the semifinals of a global competition

Main photo: Computer engineering graduate student Rui Luo operates the robot that will be entered into the September 2021 semifinals of a $10 million global robotics competition. Photo by Alyssa Stone/Northeastern University

Someday you may send a robot to attend a meeting for you. It’s like you’re there, only you’re not.

Nimble avatar robots―collaborative mechanical systems that allow a human operator in one location to feel as if they are in another location—will be the focus of an international competition that includes Northeastern’s robotics team, among 37 clubs out of 150 around the world to advance to the semifinal stage of a $10 million competition.

Members of Northeastern’s robotics team at work in the campus lab. Photo by Alyssa Stone/Northeastern University

Six members of the university’s team depart Boston on Monday for Miami, where they will compete head-to-head with clubs from Canada, Russia, Colombia, South Korea, and other nations from Sept. 11-13. The 20 best teams will share the $2 million prize. They move on to the finals in the fall of 2022 for a chance to win part of the $8 million purse.

Northeastern’s team, led by mechanical and industrial engineering professor Peter Whitney, is comprised of Stephen Alt, who is pursuing a master’s degree in computer science; Eric Schwarm, who is in his sixth year studying mechanical engineering; mechanical engineering graduate student Chunpeng Wang; computer engineering graduate student Rui Luo; and Tarik Kelestemur; who is pursuing a doctoral degree in robotics and machine learning.

Judges will be looking for robots to complete a wide range of tasks that involve social and physical interactions designed to mimic real-world scenarios, says Whitney. The tasks include attending a meeting, visiting a museum where the robot’s operator interacts physically with objects, and piecing together a puzzle.

“All of the dimensions of human-to-human interaction are up for grabs,” Whitney says.

Mechanical engineering graduate student Eric Schwarm and computer science graduate student Pushyami Kabeti adjust the hand that operates the robot. The two are part of a group of students who will be competing in Miami in September at an international robotics contest. Photos by Alyssa Stone/Northeastern University

What makes the Avatar XPrize different from other robotics competitions is that the judges, not the competing teams, put the mechanical systems through the paces. After an hour-long training period, the judges are the operators and they are the ones with whom the avatar interacts.

“So we play no role in the competition,” says Whitney. “It’s a little bit frightening to just have a one-hour period to train someone to use your system. And so that gives you kind of a sense of how easy-to-use your system needs to be.”

He points out that some of the judges will be experts in robotics, while others will have extensive experience in other areas such as virtual reality or computer systems. “And that makes it a big challenge for us to design our system, not just to be very functional for skilled operators, but it has to be very functional for newbie operators.”

One of Northeastern’s advantages heading into the competition is the frequent interactions between the hardware and software teams in the robotics lab at the Interdisciplinary Science and Engineering Complex.

Computer engineering graduate student Rui Luo puts the mechanical arm through the paces. Photo by Alyssa Stone/Northeastern University

“We have hardware and software all on the same floor and in the same lab,” says Schwarm. “We see each other every day and work with each other on everything.”

Another plus is that students on the team come from multiple research groups that are proficient in different areas, adds Whitney.

For example, in electrical and computer engineering professor Hanu Singh‘s group, they have expertise in mobile systems that are reliable and can operate under adverse communication and network quality conditions. In professor Taskin Padir’s group, they have strong knowledge of human-machine systems and system interface design.

“And then in my group we have a focus on tactile technology that gives the operator the ability to have a fine sense of touch,” says Whitney. “No one part of the system is the most important part. They all work together.”

Combining all of that proficiency and know-how gives rise to optimism that Northeastern’s mechanical arm will foster the human-to-human interaction between the person operating the robot and the human recipients of that interaction.

“Because that’s what this competition is ultimately about,” says Whitney. “It’s about human-to-human connection, not about human-to-robot connection.”


by Peter Ramjug, News @ Northeastern

Related Faculty: John “Peter” Whitney, Hanumant Singh, Taskin Padir

Related Departments:Electrical & Computer Engineering, Mechanical & Industrial Engineering