Combining maps and street level images for building height and facade estimation

ABSTRACT

We propose a method that integrates two widely available data sources, building footprints from 2D maps and street level images, to derive valuable information that is generally difficult to acquire --- building heights and building facade masks in images. Building footprints are elevated in world coordinates and projected onto images. Building heights are estimated by scoring projected footprints based on their alignment with building features in images. Building footprints with estimated heights can be converted to simple 3D building models, which are projected back to images to identify buildings. In this procedure, accurate camera projections are critical. However, camera position errors inherited from external sensors commonly exist, which adversely affect results. We derive a solution to precisely locate cameras on maps using correspondence between image features and building footprints. Experiments on real-world datasets show the promise of our method.

References

1. D. Anguelov, C. Dulong, D. Filip, C. Frueh, S. Lafon, R. Lyon, A. Ogale, L. Vincent, and J. Weaver. Google street view: Capturing the world at street level. Computer, 43(6):32--38, 2010.
2. M. Bansal and K. Daniilidis. Geometric urban geo-localization. In CVPR, 2014.
3. J. B. Burns, A. R. Hanson, and E. M. Riseman. Extracting straight lines. TPAMI, 8(4):425--455, 1986.
4. T.-J. Cham, A. Ciptadi, W.-C. Tan, M.-T. Pham, and L.-T. Chia. Estimating camera pose from a single urban ground-view omnidirectional image and a 2D building outline map. In CVPR, 2010.
5. L.-C. Chen, T.-A. Teo, C.-Y. Kuo, and J.-Y. Rau. Shaping polyhedral buildings by the fusion of vector maps and LiDAR point clouds. Photogrammetric Engineering & Remote Sensing, 74(9):1147--1157, 2008.
6. P. Cho and N. Snavely. 3D exploitationof 2D imagery. Lincoln Laboratory Journal, 20(1):105--137, 2013.
7. H. Chu, A. Gallagher, and T. Chen. GPS refinement and camera orientation estimation from a single image and a 2D map. In CVPR workshops, 2014.
8. N. M. Drawil and O. Basir. Intervehicle-communication-assisted localization. IEEE Transactions on Intelligent Transportation Systems, 11(3):678--691, 2010.
9. C. Farabet, C. Couprie, L. Najman, and Y. LeCun. Learning hierarchical features for scene labeling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8):1915--1929, 2013.
10. N. Haala and K.-H. Anders. Fusion of 2D-GIS and image data for 3D building reconstruction. International Archives of Photogrammetry and Remote Sensing, 31:285--290, 1996.
11. K. Jo, K. Chu, and M. Sunwoo. Interacting multiple model filter-based sensor fusion of GPS with in-vehicle sensors for real-time vehicle positioning. IEEE Transactions on Intelligent Transportation Systems, 13(1):329--343, 2012.
12. X. Liu and D. Wang. A spectral histogram model for texton modeling and texture discrimination. Vision Research, 42(23):2617--2634, 2002.
13. J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks for semantic segmentation. CVPR, pages 3431--3440, 2015.
14. H. Qi and J. B. Moore. Direct kalman filtering approach for GPS/INS integration. IEEE Transactions on Aerospace and Electronic Systems, 38(2):687--693, 2002.
15. F. Tack, G. Buyuksalih, and R. Goossens. 3D building reconstruction based on given ground plan information and surface models extracted from spaceborne imagery. ISPRS Journal of Photogrammetry and Remote Sensing, 67:285--290, 2012.
16. J. Tighe and S. Lazebnik. Superparsing: scalable nonparametric image parsing with superpixels. In ECCV, pages 352--365. 2010.
17. C.-P. Wang, K. Wilson, and N. Snavely. Accurate georegistration of point clouds using geographic data. In International Conference on 3D Vision-3DV, pages 33--40, 2013.
18. S. Wang, S. Fidler, and R. Urtasun. Holistic 3D scene understanding from a single geo-tagged image. In CVPR, pages 3964--3972, 2015.
19. J. Xiao, T. Fang, P. Zhao, M. Lhuillier, and L. Quan. Image-based street-side city modeling. ACM Transactions on Graphics, 28(5):114, 2009.
20. J. Yuan and A. M. Cheriyadat. Road segmentation in aerial images by exploiting road vector data. In COM. Geo, pages 16--23, 2013.
21. P. A. Zandbergen and S. J. Barbeau. Positional accuracy of assisted GPS data from high-sensitivity GPS-enabled mobile phones. Journal of Navigation, 64(03):381--399, 2011.
22. L. Zebedin, J. Bauer, K. Karner, and H. Bischof. Fusion of feature-and area-based information for urban buildings modeling from aerial imagery. In ECCV. 2008.

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision tasks