Article written by Basil Jesudason, Sharp, Robolliance Expert
In training and mentoring students on building autonomous navigation robots for competition, we need to use different sensors (ultrasonic sensors, multiple cameras, IMUs) to build a robust, elegant and effective algorithm that meets the competition use case. These are challenging problems and a very sound understanding of the mathematics is essential to have the correct implementation for your algorithm. While mentoring and training students to compete in robot competition I am interested to make sure that, while having fun building the hardware, they have a good mathematical foundation and understanding of the concepts to build upon. For example, one of the important topics is PID control.
One of challenges of developing software and algorithms for robots is to have close to a real simulated environment where you can test your code before running your software on the real vehicle. Writing your code and testing it on the simulator goes a long way towards developer productivity, especially since the software has to be written and testing done before the robot gets completely built.
Sensing and actuation are the two pillars to robotic implementations. Any type of robotic component can be surmised to fall into one of these two categories; basic activity of an autonomous robot that performs autonomous navigation consists of sensing its environment using many different types of sensors: cameras, ultrasonic sensors, LIDARs, etc.
One of basic things that I have the privilege to teach the team developing the software and algorithms for the competition robot is how to implement a PID controller in code. PID stands for “proportional, integral, derivative.” These three terms describe the basic elements of a PID controller. Each of these elements performs a different task and has a different effect on the functioning of a system.
For the robot that the team built, the challenges included going over obstacles that would make your average road pot-hole look like child’s play. The goal is maintaining robot stability both in terms of its orientation and at the same time maintain a constant speed. While the proportional part of PID control was fairly straightforward for the students to grasp, the concept and the math behind the integral and derivative were difficult to grasp.
As a roboticist, while these are concepts that I have been implementing in code all my life, the trick was to explain these concepts in ways that made sense to a typical, over-achieving science and math junior. It is indeed a delight to me when they finally get it and their eyes light up when they are able to turn their calculus knowledge into actual practical real work example that they can put in practice.
While manipulators or robot arms have traditionally been used in the factory floor, mobile manipulators (or robot arms attached to an UGV) are an important addition. We not only want the robot to collect data as it goes about its tasks, we also would like for it to do things or change the environment in a specific manner: pick an object and examine it or open a door or even remove an obstacle.
Most manipulators these days have at least three degrees of freedom or joints. For the robot competition the attached manipulator was tasked with opening an arched gateway called portcullis that has a door that opens when lifted up. The software task is to algorithmically compute in code the joint angles that the manipulators joints need to achieve in order for the “end effector” or the gripper to grab and lift the door. Here we benefited from the knowledge of PID control algorithms we discussed earlier. But the real winner is what is often known in robotics as the IK algorithm (which stands for Inverse Kinematics)
In this illustration the algorithm needs to find the angles Q1 and Q2.
PID control goes into parts of a larger algorithm on their robot that I have spent many weeks helping them build out to take to the competition. In the competition, one of the objectives is to detect and avoid obstacles. They must also find goals that they can shoot boulders at using actuators. Detecting obstacles and boulders was achieved using computer vision algorithms that use a monocular IP camera which was mounted on the robot.
The robot that the team built for the competition is a perfect example of a real world commercial robot where it needs to navigate autonomously around obstacles and perform various tasks with an attached manipulator.
Again, as a professional, I use computer vision to develop vision-based navigation algorithms. The trick was in coming up with the right algorithm. After detailed discussions, we split tracking and mapping into two separate tasks, processed in parallel threads on board the robot: one thread deals with the task of robustly tracking robot motion, while the other produces a 3D map of point features from previously observed video frames. This allows the use of computationally expensive batch optimization techniques not usually associated with real-time operation. The result is a system that produces detailed maps with thousands of landmarks which can be tracked at frame-rate, with an accuracy and robustness rivalling that of state-of-the-art model-based systems.
The majority of vision-based robot navigation systems operate with prior knowledge of the robot’s environment - i.e., some form of map. This could be a map of a city, CAD model of a component requiring maintenance, or even a sparse map of fiducials known to be present in the scene usually created by manually driving the robot with a camera to build the map. The UI application then allows the user to interact with this environment based on prior information on salient parts of this model (e.g. “Here’s a curb I want to avoid” or “here’s the parked truck I don’t want to hit”). If the map or model provided is complete, then the robot autonomously navigates what is tracked through the camera can be registered to areas in the map. In the case of the robot competition field, this is not possible since obstacles that the robot needs to traverse are all not known prior to the match. So the robot has to build a map as it goes along in the initial seconds of the start of the game. Once the map is built then it can be used to traverse the obstacles and reach points on the field where it can score points. Here again vision tracking is used to score points.
One of things that the robot in the competition needed to do is to allow human drivers to tele-operate the robot. The human drivers are often not able to see the robot since they are outside the field and located on the opposite side of where the robot frequently is. This means that they have to tele-operate the robot by looking at the live video stream that needs to be streamed. This means that the video stream has to have low-latency.
In a field where there are five other robots at the same time, low-latency is often a challenge since the bandwidth available is very limited: it is shared among five other robots. Also, the team is only legally allowed to use 2.4 Ghz. Hence, optimizing the video stream by choosing the right protocols is critical. While MPEG is commonly used in free video streaming software tools, the team needed a more optimized and adaptive protocol that provided a low-latency video stream that the team can rely on for the time critical teleoperation part of the competition. Since the IP camera we used on the robot did the heavy lifting of encoding the video stream and implemented an RTSP server, we opted for a solution using RTSP for live video streaming.
Other sensors that were fused into the software stack that was built for the robot included ultrasonic sensors and gyros. Unfortunately, these were low-cost sensors and they were prone to errors: the gyro we used drifted quite a bit and frequently required recalibration and the ultrasonic sensors were prone to errors and false-positive. The students were thrilled to learn to implement a filter in code to filter out the noise and errors from these sensors.
As a roboticist, while my life-long passion has been autonomous navigation of ground vehicles, I must admit that lately I have taken an interest in autonomous navigation of MAVs (micro aerial vehicles). These flying robots are ubiquitous and manually flying them is a common activity. My interest is to implement autonomous navigation of these MAVs that often do not contain GPS and need to navigate with just a camera and on-board sensors with limited computing capability. The key challenges here are to localize the robot purely from its own sensor data and to robustly navigate it even under temporary sensor outage. This requires both a solution to the so-called simultaneous localization and mapping (SLAM) problem as well as robust state estimation and control methods. These challenges are even more pronounced on low-cost hardware with inaccurate actuators, noisy sensors, signiﬁcant delays, and limited on-board computation resources. In the next edition of this series, we will see what it takes to achieve robust navigation on these flying robots.