Developing Intelligent Vision
Humans are almost entirely reliant on their senses. Without external input from the outside world, we are unable to locate, identify, navigate, operate, and much more. Similarly, robots require advanced perception to function in a world designed for the sensory aptitude of humans. However, while vision is an inexpensive sensing system for robots, it is prone to errors for various causes such as reflective surfaces, blur, and texture-less scenes. A research duo from the College of Information and Computer Sciences at the University of Massachusetts Amherst, Sadegh Rabiee (PhD student) and Joydeep Biswas (Assistant Professor) are using Jackal UGV to train competency-aware vision-based obstacle avoidance systems that can predict such failures. This would allow for less expensive and safer robot deployments. It is only through learning competence-aware perception algorithms that one can predict their failure cases and reason about their types of failure. Therefore, their Introspective Vision for Obstacle Avoidance (IVOA) project focuses on learning the competency of computer vision algorithms for the task of obstacle avoidance.
Giving Jackal UGV Its Eyes
To prepare Jackal UGV for the task, Rabiee and Biswas equipped the platform with a pair of stereo cameras, as well as a depth camera to provide sparse ground truth. These would be used to log RGB and depth images at full frame rate during deployments of the robot. Those images are then separately processed for obstacle avoidance. When the plans generated by the two disagree, and when the environment is known to be one where the depth sensor is more reliable, the point where the paths diverge is projected onto the image plane of the RGB image. Consequently, an image patch centered at that location is extracted as an example of unreliable image conditions. By utilizing such images patches that yield failures of the image-based algorithm, the team can train an introspection model to learn to predict the following:
- Will an input image yield failures?
- Which parts of the image are likely to cause such failures?
- What type of failure is expected (false positive vs. false negative)?
- How many distinct classes of failures were extracted from the training set?
Tailoring Jackal UGV for research in long-term autonomy for mobile robots required the UMass team to customize the platform with their own desired sensors and payloads. To begin, they attached a stereo pair of Point Grey cameras for vision-based SLAM and obstacle avoidance, as well as Kinect depth sensors to provide ground truth for their vision-based obstacle avoidance system. Next, a Velodyne Lidar (VLP 16) was used for research on lidar-based SLAM, as well as ground truth generation for vision-based SLAM. The final pieces of the hardware puzzle included a touch screen monitor for human-robot interaction and an Intel NUC for additional computing power. The touch screen enables quick and subtle debugging, as well as allowing the robot to interact with humans via a graphical user interface. An example of this would be a robot seeking help from people nearby for using the elevator. In terms of software, the IVOA team developed their own complete ROS-based stack for autonomous navigation of the robot, including SLAM implementation, obstacle avoidance, and planning.
Fitting Clearpath Into the Equation
“Clearpath Robotics makes research in robotics easier via providing reliable robot platforms that are easy to customize. Also, we have found the support team at Clearpath to be very responsive and helpful”.
Where does Clearpath Robotics fit in? Well, IVOA’s data collection process requires having a mobile robot platform capable of operating reliably for extended periods of time in different environments and traversing different types of terrain. At the same time, IVOA relies on extensive self-labeled training data, which would have been infeasible to collect manually. Using Jackal UGV enabled the acceleration of their research process in three ways:
- Eliminating the need to build and maintain their own UGV. This would have required a significant amount of time and effort
- Providing a flexible, modular platform with hardware and software support to easily experiment with different sensor configurations
- Proving to be a robust, low-maintenance research platform
The IVOA team were already confident in using Jackal UGV as they had done so in the past for their project titled “A Friction-Based Kinematic Model for Skid-Steer Wheeled Mobile Robots”, which was published at ICRA 2019 (IEEE International Conference on Robotics and Automation). The researchers chose Jackal UGV because it was a good fit for developing their algorithms and eliminated the need to design and build their own robot, ultimately allowing for a smooth experience when conducting experiments for their research. “Clearpath Robotics makes research in robotics easier via providing reliable robot platforms that are easy to customize”, says Sadegh Rabiee, “Also, we have found the support team at Clearpath to be very responsive and helpful”.
Looking Ahead
The IVOA project continues to be improved upon and the team has plans to extend their work on competence-aware perception systems to more perception problems such as vision-based simultaneous localization and mapping (SLAM). The end goal is to deploy their autonomous robots at the university campus scale and have them perform safe navigation via competence-aware vision systems. Currently, they have been able to successfully introduce IVOA, an architecture for self-aware stereo vision-based obstacle avoidance systems capable of predicting their failures, while distinguishing between false positive and false negative instances. This project will also appear at IROS 2019 (IEEE/RSJ International Conference on Intelligent Robots and Systems).
To learn more about Jackal UGV, click here.
To learn more about the University of Massachusetts Amherst’s College of Information and Computer Sciences, click here.
To access the full IVOA paper, click here.