High speed obstacle avoidance using monocular vision and reinforcement learning

ABSTRACT

We consider the task of driving a remote control car at high speeds through unstructured outdoor environments. We present an approach in which supervised learning is first used to estimate depths from single monocular images. The learning algorithm can be trained either on real camera images labeled with ground-truth distances to the closest obstacles, or on a training set consisting of synthetic graphics images. The resulting algorithm is able to learn monocular vision cues that accurately estimate the relative depths of obstacles in a scene. Reinforcement learning/policy search is then applied within a simulator that renders synthetic scenes. This learns a control policy that selects a steering direction as a function of the vision system's output. We present results evaluating the predictive ability of the algorithm both on held out test data, and in actual autonomous driving experiments.

References

1. Barron, J., Fleet, D., & Beauchemin, S. (1994). Performance of optical flow techniques. Int'l Journal of Computer Vision, 12, 43--77.
2. Blthoff, I., Blthoff, H., & Sinha, P. (1998). Top-down influences on stereoscopic depth-perception. Nature Neuroscience, 1, 254--257.
3. Criminisi, A., Reid, I., & Zisserman, A. (2000). Single view metrology. Int'l Journal of Computer Vision, 40, 123--148.
4. Davies, E. (1997). Machine vision: Theory, algorithms, practicalities 2nd ed. Academic Press.
5. Gini, G., & Marchi, A. (2002). Indoor robot navigation with single camera vision. Proc. Pattern Recognition in Information Systems, PRIS, Spain.
6. Honig, J., Heit, B., & Bremont, J. (1996). Visual depth perception based on optical blur. Proc. of Int'l Conf. on Image Processing (pp. 721--724).
7. Huber, P. (1981). Robust statistics. New York: Wiley.
8. Jahne, B., & Geissler, P. (1994). Depth from focus with one image. Proc. IEEE Conf. on Computer Vision and Pattern Recognition CVPR (pp. 713--717).
9. Kardas, E. (2005). Monocular cues in depth perception. {Online}: http://peace.saumag.edu/faculty/Kardas/Courses/GP Weiten/C4SandP/MonoCues.html.
10. Kearns, M., & Singh, S. (1999). Finite-sample rates of convergence for q-learning and indirect methods. NIPS 11 (pp. 996--1002). The MIT Press.
11. Kim, M., & Uther, W. (2003). Automatic gait optimisation for quadruped robots. Proc. Australasian Conf. on Robotics and Automation (pp. 1--9).
12. Klarquist, W., Geisler, W., & Bovik, A. (1995). Maximum-likelihood depth-from-defocus for active vision. Proc. Int'l Conf. on Intelligent Robots and Systems (pp. 374--379).
13. Kohl, N., & Stone, P. (2004). Policy gradient reinforcement learning for fast quadrupedal locomotion. Proc. IEEE Int'l Conf. Robotics and Automation.
14. Kudo, H., Saito, M., Yamamura, T., & Ohnishi, N. (1999). Measurement of the ability in monocular depth perception during gazing at near visual target-effect of the ocular parallax cue. Proc. IEEE Int'l Conf. Systems, Man & Cybernetics (pp. 34--37).
15. LeCun, Y. (2003). Presentation at Navigation, Locomotion and Articulation workshop. Washington DC.
16. Loomis, J. M. (2001). Looking down is looking up. Nature News and Views, 414, 155--156.
17. Nagai, T., Naruse, T., Ikehara, M., & Kurematsu, A. (2002). Hmm-based surface reconstruction from single images. Proc. IEEE Int'l Conf. on Image Processing (pp. II--561 -- 11--564).
18. Ng, A. Y., & Jordan, M. (2000). Pegasus: A policy search method for large mdps and pomdps. Proc. 16th Conf. UAI.
19. Pomerleau, D. (1989). An autonomous land vehicle in a neural network. NIPS 1. Morgan Kaufmann.
20. Scharstein, D., & Szeliski. R. (2002). A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int'l Journal of Computer Vision, 47, 7--42.
21. Shao, M., Simchony, T., & Chellappa, R. (1988). New algorithms from reconstruction of a 3-d depth map from one or more images. Proc. IEEE Conf. on Computer Vision and Pattern Recognition CVPR (pp. 530--535).
22. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning. MIT Press.
23. Wu, B., Ooi, T. L., & He, Z. J. (2004). Perceiving distance accurately by a directional process of integrating ground information. Letters to Nature, 428, 73 77.

• Computing methodologies