Friday, April 11, 2008

My disobedient robot

My senior project this semester has been to build an autonomous robot racer with a team of four other people. Translated into English, that means that we take a remote control toy monster truck, stick a computer and camera on it, and make it drive around all by itself. It's been a lot of fun—and a lot of work. We had our final competition yesterday and fared horribly.

The goal of the project was to get our robot to use computer vision to drive a race course all by itself. The course was a bunch of orange and green pool noodles (we call them "pylons" to sound sophisticated) that were set up vertically and spaced out on the floor. Orange meant that the robot had to pass them on the right and then turn to the left; green meant the opposite. The robot was supposed to use its camera to find a noodle, drive to it, make a turn, and repeat until it had finished the course.

That was the idea, anyway. None of the five teams that competed in the project were completely successful. Our team fared the worst.

There are several components to a system that can autonomously control a racer:
  • A color segmenter that can identify areas in the image that are the specific shade of green or orange that we're looking for.
  • A feature extractor that can take those areas and identify the precise position of the pylon.
  • A vision-guided control system that uses the data from the feature extractor to control the robot's steering and throttle to drive toward pylons.
  • A dead reckoning control system that "drives blind" to control the robot while pylons are out of view.
  • A "mission control" computer program that runs on a desktop computer that communicates wirelessly with the robot. This program doesn't control the robot; it's just used to monitor its state, fine-tune parameters, and send an emergency stop command if the robot gets out of control.
Our robot could (mostly) successfully drive toward pylons, but it had a really hard time correctly executing turns. We got it working a week ago on a simple two-pylon course, but that's about the most advanced thing it could do. It was pretty disobedient, and our competition yesterday was an embarrassment.

What went wrong? A lot of things. Most of it boils down to poor planning and a rushed schedule.
  • We optimized prematurely. We tried to implement our feature extractor in hardware (VHDL) because we wanted it to run fast. Developing hardware is a lot trickier than developing software, and we spent three weeks trying to get it to work. We finally decided to cut our losses, and I wrote the object extractor in C and had it working in two hours. In hindsight, I should have written it in C to begin with, and only converted it to hardware if it was too slow. As it turns out, the software, running at 100MHz, was plenty fast enough to process images at the camera's full 35 fps frame rate.
  • We added too much complexity too quickly. When we finally got our feature extractor working a couple of weeks ago, we tried to throw everything else into the pot all at once. When you add a lot of complexity all at the same time, it's hard to tell where to look when something doesn't work. Instead, we should have tested each feature independently. When we were confident that a feature was working by itself, we could add it to the rest of the system and make sure that it works when integrated. That way it's a lot easier to isolate problems.
  • We didn't have a consistent schedule. With a semester-long project like this, it's easy to put it off and work on more pressing assignments for other classes. That hurt us in the long run, though. There were also several times when one person was assigned to complete an important part of the project by a certain day, and they didn't have it done. That then delayed everyone else who couldn't make progress until that part was in place.
  • It was hard to work in parallel. We had only a single development computer, so only one or two people could work on the project at once. That wasn't an issue for most of the semester, but when it was crunch time, it would have been really helpful to have two or three workstations.
  • Attitude. Most school project are very well-defined and self-contained. This project was different: it was very open ended, both in approach and schedule. This type of project requires more discipline and more creativity than typical school projects. It also requires a different attitude. When something doesn't work—and it seems that nothing works on the first try—you can either give up and say "I tried", or you can buckle down, try a different approach, do research, talk to others, and make things happen, even though the problem is hard. Some members of our team were better than others at taking the reins and making things happen. That's not mean to bash my team members, though; I feel like everyone contributed to meaningfully to the project.
I feel like with another day or two of work, we could have our racer driving courses pretty well. It's a shame that we didn't finish in time. However, I think that everyone on our team learned a lot, and the project was definitely worth doing. We accomplished in the last three weeks what took the other teams most of a semester to accomplish. I think that we have about 90% of the functionality of most other teams, but that last 10% is key to having a working robot.

I'm really happy with the work that everyone on our team did. I'm also really happy that I'll be able to get a little sleep again. :)


Morebadger said...

YEAH for SLEEP! who whap!

jacob said...

Learning from a major disappointment like this can be just as educational as learning for a huge success. Its just not as fun. But you can still go program ipods for apple or whatever and be just as good as an engineer.

jacob said...

It seems that the hardest part of this project is constructing a "world view" based on limited imaging.

I think the goal would be more achievable if you had three cameras: a camera looking to the left for orange pylons, a camera looking to the right for green pylons, and a camera looking forward for any pylon. You could use the forward camera to get you close to the pylon, and then a side camera to help you maneuver around it.