Creating and integrating fully autonomous robots is a long process with significant periods of trial and error. In lieu of these robots handling many of society’s needs, currently, semi-autonomous teleoperated robots fill in instead. Teleoperation means the remote control of a robot through the use of an interface in which a human user can input commands that the robot will complete. In fact, teleoperation is a great approach to keep humans safer overall.
Bridging the Gap Between Operators and Robots
But, the field of robot teleoperation is not perfect. An operator continuously faces challenges to understand the remote information they’re receiving, must issue appropriate commands, and evaluate their performance in real-time. This highlights the importance of effective and functional teleoperation interfaces, which are the only gateways to connect operators to the remote world. A joint team from the Human-Robot Interaction Laboratory at the University of Manitoba is using Jackal UGV to identify key teleoperation challenges and present state-of-the-art interface designs so that the robotics community can learn the importance of and work on improving teleoperation interfaces.
The Human-Robot Interaction Lab is recognized as an international leader in the field of human-computer interaction. Their team tackles a range of problems surrounding how people can better work with robots, and in particular, investigates how to improve the design of teleoperation interfaces to reduce operator mistakes, cognitive load, and increase overall operational effectiveness and efficiency.
One of the main challenges when considering robot teleoperation and developing solutions to address these challenges is exploring teleoperation interfaces to support operators’ awareness. That is, stable connections between operators and robots in real-time are always challenging. Another challenge is to maintain a robot’s stability. For example, if the robot’s locomotion makes the cameras shake, the interface presents a shaky video feed with stable widgets, which present information supporting the operators. This increases operator fatigue and motion sickness, which can detract from one’s intended research.
Jackal UGV as a Valuable Sensor Hub
For HRI’s teleoperation project, Jackal UGV collected remote information using its sensors and then streamed it directly to a remote operator. The remote operator would watch the video feed, listen to the audio feed, and issue commands to the robot. The robot would then move based on those received commands. The team’s interface connected to the robot to retrieve the remote information and presented it to the operator in real-time.
“During our study, we set up a mock search and rescue environment where the robot could push over almost all furniture without any issue. Sometimes, I had to hold the furniture up with my whole body and I still failed. I was very busy during the experiments to fix the environment. Some people did not even notice that they broke my setting because the robot smoothly pushed away things.”
– Stela H. Seo, Assistant Professor (Kyoto University)
For each internal project, they had different focuses. For one project, the HRI Lab tried to provide general robot states of team robots in real-time teleoperation. In another project, the team created a social agent which influences people’s teleoperation behaviors. For each of these projects, Jackal UGV was able to provide the necessary remote sensor information and respond to all commands from the team’s teleoperation interfaces. To ensure the best sensor information, the HRI Lab integrated a Pointgrey camera. The original PTZ security camera was an issue for the project as the participants used the robot very roughly (unintentionally) and the security camera could not handle all the wobbly shakes. Inevitably, the camera would point to the wrong place from time to time (like inside of the housing unit). Thus, they made the switch to a Pointgrey camera – a swap that was made simple by Jackal UGV’s modular nature
To take some of the potential issues out of the testing phase, the HRI Lab opted to utilize Jackal UGV as it is an affordable yet robust and powerful mobile robot base. Specifically, it provided the team with a stable wireless connection and stable locomotion with high torque. As well, the robot was very flexible in terms of attaching or removing devices which made hardware or mounting changes during the course of testing much easier. Without Jackal UGV on hand, the team would have resorted to building their own robot using toolkits such as the bioloid kit or dynamixel servos. This, however, would have been a lengthy process requiring non-research focused work which the team ultimately wanted to avoid. As well, writing reliable control software can be a significant challenge for many researchers and would have diverted more valuable time and effort from the team.
Ultimately, the HRI team opted for Jackal UGV because it provided three core benefits:
- Torque
- Processing
- Battery Life
As Assistant Professor from the Kyoto University , Stela H. Seo explains: “During our study, we set up a mock search and rescue environment where the robot could push over almost all furniture without any issue. Sometimes, I had to hold the furniture up with my whole body and I still failed. I was very busy during the experiments to fix the environment. Some people did not even notice that they broke my setting because the robot smoothly pushed away things.” In terms of processing, the team noted that their background software, which was not particularly optimized, ran very smoothly without any issues on Jackal UGV. As well, even with all of the background processes and continuously driving, the robot had a long run-time. The battery duration was particularly good for their back-to-back experiments.
Through the Jackal UGV project, the HRI Lab was able to present a novel approach and a set of visual widgets to represent the general robot states of team robot members in multi-robot teleoperation. With the results from their designs using animation, color, and emojis, and experiments, the team provided a list of design insights to the teleoperation interface design community. This was possible because the robot held through the rough uses in their series of user experiments
This Jackal UGV teleoperation project is but one of many efforts from the HRI Lab to innovate the field and create solutions to aid other researchers in their work. However, the team also has ambitious future goals. They are interested in taking on new challenges such as handling a large scale in a teleoperation interface such as through multiple robots or multiple sensors. In contrast, for this project, the team had a limited number of (simulated) team robots and a limited number/type of remote sensor inputs.
This joint project team was composed of Stela H. Seo (Assistant Professor, Kyoto University), Daniel J. Rea (Assistant Professor, University of New Brunswick), and James E. Young (Professor, University of Manitoba).
To learn more about the Human-Robot Interaction Lab, visit their website here.
To learn more about Jackal UGV, visit our website here.