The goal of the project was to get our robot to use computer vision to drive a race course all by itself. The course was a bunch of orange and green pool noodles (we call them "pylons" to sound sophisticated) that were set up vertically and spaced out on the floor. Orange meant that the robot had to pass them on the right and then turn to the left; green meant the opposite. The robot was supposed to use its camera to find a noodle, drive to it, make a turn, and repeat until it had finished the course.
That was the idea, anyway. None of the five teams that competed in the project were completely successful. Our team fared the worst.
There are several components to a system that can autonomously control a racer:
- A color segmenter that can identify areas in the image that are the specific shade of green or orange that we're looking for.
- A feature extractor that can take those areas and identify the precise position of the pylon.
- A vision-guided control system that uses the data from the feature extractor to control the robot's steering and throttle to drive toward pylons.
- A dead reckoning control system that "drives blind" to control the robot while pylons are out of view.
- A "mission control" computer program that runs on a desktop computer that communicates wirelessly with the robot. This program doesn't control the robot; it's just used to monitor its state, fine-tune parameters, and send an emergency stop command if the robot gets out of control.
What went wrong? A lot of things. Most of it boils down to poor planning and a rushed schedule.
- We optimized prematurely. We tried to implement our feature extractor in hardware (VHDL) because we wanted it to run fast. Developing hardware is a lot trickier than developing software, and we spent three weeks trying to get it to work. We finally decided to cut our losses, and I wrote the object extractor in C and had it working in two hours. In hindsight, I should have written it in C to begin with, and only converted it to hardware if it was too slow. As it turns out, the software, running at 100MHz, was plenty fast enough to process images at the camera's full 35 fps frame rate.
- We added too much complexity too quickly. When we finally got our feature extractor working a couple of weeks ago, we tried to throw everything else into the pot all at once. When you add a lot of complexity all at the same time, it's hard to tell where to look when something doesn't work. Instead, we should have tested each feature independently. When we were confident that a feature was working by itself, we could add it to the rest of the system and make sure that it works when integrated. That way it's a lot easier to isolate problems.
- We didn't have a consistent schedule. With a semester-long project like this, it's easy to put it off and work on more pressing assignments for other classes. That hurt us in the long run, though. There were also several times when one person was assigned to complete an important part of the project by a certain day, and they didn't have it done. That then delayed everyone else who couldn't make progress until that part was in place.
- It was hard to work in parallel. We had only a single development computer, so only one or two people could work on the project at once. That wasn't an issue for most of the semester, but when it was crunch time, it would have been really helpful to have two or three workstations.
- Attitude. Most school project are very well-defined and self-contained. This project was different: it was very open ended, both in approach and schedule. This type of project requires more discipline and more creativity than typical school projects. It also requires a different attitude. When something doesn't work—and it seems that nothing works on the first try—you can either give up and say "I tried", or you can buckle down, try a different approach, do research, talk to others, and make things happen, even though the problem is hard. Some members of our team were better than others at taking the reins and making things happen. That's not mean to bash my team members, though; I feel like everyone contributed to meaningfully to the project.
I'm really happy with the work that everyone on our team did. I'm also really happy that I'll be able to get a little sleep again. :)