It’s a common issue faced in robotics research and design; how can we create robots, particularly mobile manipulators that can work alongside humans in highly dynamic and unpredictable environments? The problem is that right now, mobile manipulators are simply not ready for such a deployment. It is not possible to program a robot for everything that might happen and it is very hard to perform a variety of complex tasks while still maintaining a high degree of reliability.
However, a combined team from the AIRLab (AI for Retail Lab) and Delft University of Technology (TU Delft) are looking to address these challenges, specifically by redefining retail through technology. Using Boxer, their goal is to research mobile manipulation (i.e. a mobile base with one or more robot arms) with an application to retail environments outside controlled laboratories. In this aim, Boxer is aiding graduate academic research at TU Delft, exploring the mobility of a robot arm in human-shared environments (i.e. a mockup retail store) to reach more locations and perform different and complex tasks such as mobile pick and place.
The Hunt for Dynamism
The team’s main goal is to push the boundaries of robotics and bring mobile manipulation outside of controlled labs. More specifically, they aim at developing new methods for mobile manipulation so that robots can perform long missions with a high degree of explainability, move intelligently and safely in the environment, and respect ethical concerns.
But, there are several challenges facing them. On the algorithmic side, the problem of navigation, perception, and manipulation is not yet fully solved to the degree needed to work dynamically alongside humans. Things happen fast in the environment and a robot needs to understand its surroundings and react to changes. Humans do this pretty easily but they rely on several years of experience and common sense knowledge which is still hard to reproduce artificially in a reliable manner.
There is also the problem of integrating all the different components (hardware and software) in a way where they work together effectively and consistently. The general problem of mobile manipulation is split into task planning, motion planning, and manipulation using a mobile base with a robot arm. All the different algorithms need to communicate correctly with each other and the hardware architecture needs to support fast and reliable communication and motions.
“Boxer is very robust and powerful which allowed us to focus on developing our algorithms for motion versus programming the dynamics of the (moving) arm.”
– Ph.D. Candidate, Corrado Pezzato
Boxer For Easy Integration
To address several of these issues, the team used Boxer as a sturdy mobile base that was both powerful and could move quickly. Further, its plug-and-play compatibility with third-party sensors and components and its customizable nature made safely mounting the project’s Franka Emika Panda easy, as well as all the required sensors for navigation in their test lab. This combined integration with Boxer provided for a more advanced user interface of the whole system, which was particularly important for the hardware and software integration of the team’s different systems. The platform set-up included a LiDAR, wheel encoders, and the IMU, as well as utilizing the available sensors on Boxer (primary LIDAR 10m range 260° FOV, forward stereo camera system, rear-facing sonar, and odometry). The total payload was ~25kg, including arm, controllers, and accessories for communication.
The AIRLAB enjoyed the quick start-up time for their project, since Boxer was readily available in their lab, and the robot arm was already mounted on the base and connected with the robotic platform. Furthermore, Ph.D. candidate Corrado Pezzato found the base to be: “very robust and powerful which allowed us to focus on developing our algorithms for motion versus programming the dynamics of the (moving) arm.” The team had a reliable base in Boxer and could focus on what really mattered the most to them: developing their algorithms.
Enabling Research & Development
The Clearpath Boxer was ultimately involved in two different research topics. One was whole-body control using model predictive control (MPC) such that the base and arm could be controlled simultaneously for more advanced motions. The other was to illustrate the symbolic decision-making at the task level using behavior trees and active inference. This was done to decide which and when motions using the MPC should be executed and why. Thus, the combined mobile manipulator project was able to successfully showcase two different algorithms, as well as demonstrate the interconnection and integration of these two research projects.
But the work isn’t over yet. The AIRLab team is still actively working on further developing the integration of the hardware and software for an improved interface for researchers. They are also consolidating the current architecture and solution as a baseline for future custom algorithms and comparisons with other mobile manipulators. As well, their work has also already been published into two separate papers, each exploring one of the two different algorithms: “Active Inference and Behavior Trees for Reactive Action Planning and Execution in Robotics” and “Coupled Mobile Manipulation via Trajectory Optimization with Free Space Decomposition” (no link yet).
The AIRLab and TU Delt collaborative team consists of Team Lead and Technical Support Stefan Bonhof and Ph.D. candidates Corrado Pezzato, Max Spahn, and Mert Mire.
You can learn more about AIRLab’s project on their website here.
You can learn more about Boxer on our website here.