Driving in an augmented-city: from fast and automatic large scale environment modeling to on-line 6DOF vehicle localization

ABSTRACT

To provide high quality Augmented Reality service in a car navigation system, accurate 6DoF localization is required. To ensure such accuracy, most of current vision-based solutions rely on an off-line large scale modeling of the environment. Nevertheless, while existing solutions require expensive equipments and/or a prohibitive computation time, we propose in this paper a complete framework that automatically builds an accurate city scale database of landmarks using only a standard camera, a GPS and Geographic Information System (GIS). As illustrated in the experiments, only few minutes are required to model large scale environments. Then, a localization algorithm can use the resulting databases for a high quality Augmented Reality experiences.

References

1. Agarwal, S., Furukawa, Y., Snavely, N., Simon, I., Curless, B., Seitz, S. M., and Szeliski, R. 2011. Building rome in a day. Commun. ACM 54, 10, 105--112.
2. Arth, C., Wagner, D., Klopschitz, M., Irschara, A., and Schmalstieg, D. 2009. Wide area localization on mobile phones. In ISMAR.
3. A. Taneja, L. Ballan, and M. Pollefeys. 2012. Registration of spherical panoramic images with cadastral 3d models. In 3DIMPVT.
4. Frahm, J. M., Fite-Georgel, P., Gallup, D., Johnson, T., Raguram, R., Wu, C., Jen, Y. H., Dunn, E., Clipp, B., Lazebnik, S., and Pollefeys, M. 2010. Building rome on a cloudless day. In ECCV.
5. Gay-Bellile, V., Lothe, P., Bourgeois, S., Royer, E., and Naudet-Collette, S. 2010. Augmented reality in large environments: Application to aided navigation in urban context. In ISMAR.
6. H. Lim, Sinha, S. N., Cohen, M. F., and Uyttendaele, M. 2012. Real-time image-based 6-dof localization in large-scale environments. In CVPR.
7. Irschara, A., Zach, C., Frahm, J., and Bischof, H. 2009. From structure-from-motion point clouds to fast location recognition. In CVPR.
8. Kaminsky, R., Snavely, N., Seitz, S., and Szeliski, R. 2009. Alignment of 3d point clouds to overhead images. In CVPR Workshop.
9. Lanaout, D., Gay-Bellile, V., Bourgeois, S., and Dhome, M. 2013. Vehicle 6-dof localization based on slam constrained by gps and digital elevation model information. In ICIP To Appear.
10. Lhuillier, M. 2012. Incremental fusion of structure-from-motion and gps using constrained bundle adjustments. IEEE Transactions on Pattern Analysis and Machine Intelligence 34, 12, 2489--2495.
11. Li, Y., Snavely, N., Huttenlocher, D., and Fua, P. 2012. Worldwide pose estimation using 3d point clouds. In ECCV.
12. Lothe, P., Bourgeois, S., Dekeyser, F., Royer, E., and M. Dhome. 2009. Toward large scale model construction for vision-based global localisation. In VISAPP.
13. Mouragnon, E., Lhuillier, M., Dhome, M., Dekeyser, F., and Sayd, P. 2006. Real time localization and 3d reconstruction. In CVPR.
14. Pylvanainen, T., Roimela, K., Vedantham, R., Itaranta, J., and Grzeszczuk, R. 2010. Automatic alignment and multi-view segmentation of street view data using 3d shape prior. In 3DPVT.
15. Strecha, C., Pylvänäinen, T., and Fua, P. 2010. Dynamic and scalable large scale image reconstruction. In CVPR.
16. Tamaazousti, M., Gay-Bellile, V., Naudet-Collette, S., Bourgeois, S., and Dhome, M. 2011. Nonlinear refinement of structure from motion reconstruction by taking advantage of a partial knowledge of the environment. In CVPR.
17. Wang, C., Wilson, K., and Snavely, N. 2013. Accurate georegistration of point clouds using geographic data. In 3DV.
18. Zamir, A. R., and Shah, M. 2010. Accurate image localization based on google maps street view. In ECCV.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision problems

      • Computer vision tasks

        • Scene understanding

  • Machine learning