3DLoc: 3D Features for Accurate Indoor Positioning

Abstract

A variety of indoor applications require both accurate location and orientation, such as indoor navigation and augmented reality. This paper presents 3DLoc, with which you can find your location and orientation by pointing your smartphone camera at 3D features e.g., doors and entrances. Different from the previous image-based localization of matching features via SIFT or SURF, 3DLoc takes advantage of rules for 3D features, including the ratio between height and width, the orientation and the distribution on the 2D floor map. The features around users are regarded as a unique 3D signature for the location. Based on prior researches on vanishing points and indoor geometric reasoning, we propose an algorithm to extract the signature from captured images and robustly decode the signature to accurate location and orientation. In terms of efficiency and user-friendliness, a series of optimizations are adopted through fusion of smartphone sensors and vision. We conduct experiments on different floors of a typical office building via the prototype built on Huawei P7 and iPhone 5S. Ninety percent of errors for location and orientation are within 25cm and two de4rees, respectively. With a 2D floor map provided, KB (-KiloByte-) level storage is required for the additional 3D information.

References

1. Sameer Agarwal, Yasutaka Furukawa, Noah Snavely, Ian Simon, Brian Curless, Steven M Seitz, and Richard Szeliski. 2011. Building rome in a day. Commun. ACM 54, 10 (2011), 105--112.
2. Cuneyt Akinlar and Cihan Topal. 2011. EDLines: A real-time line segment detector with a false detection control. Pattern Recognition Letters 32, 13 (2011), 1633--1642.
3. Paramvir Bahl and Venkata N Padmanabhan. 2000. RADAR: An in-building RF-based user location and tracking system. In INFOCOM 2000. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies. Proceedings. IEEE, Vol. 2. Ieee, 775--784.
4. Paramvir Bahl, Venkata N Padmanabhan, and Anand Balachandran. 2000. Enhancements to the RADAR user location and tracking system. Microsoft Research 2, MSR-TR-2000--12 (2000), 775--784.
5. Herbert Bay, Tinne Tuytelaars, and Luc Van Gool. 2006. Surf: Speeded up robust features. Computer vision--ECCV 2006 (2006), 404--417.
6. Jean-Charles Bazin and Marc Pollefeys. 2012. 3-line RANSAC for orthogonal vanishing point detection. In Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on. IEEE, 4282--4287.
7. Jean-Charles Bazin, Yongduek Seo, Cédric Demonceaux, Pascal Vasseur, Katsushi Ikeuchi, Inso Kweon, and Marc Pollefeys. 2012. Globally optimal line clustering and vanishing point estimation in manhattan world. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on. IEEE, 638--645.
8. Ionut Constandache, Romit Roy Choudhury, and Injong Rhee. 2010. Towards mobile phone localization without war-driving. In Infocom, 2010 proceedings ieee. IEEE, 1--9.
9. Erick Delage, Honglak Lee, and Andrew Y Ng. 2007. Automatic single-image 3d reconstructions of indoor manhattan world scenes. In Robotics Research. Springer, 305--321.
10. Wael Elloumi, Sylvie Treuillet, and Rémy Leconge. 2014. Real-time camera orientation estimation based on vanishing point tracking under Manhattan World assumption. Journal of Real-Time Image Processing (2014), 1--16.
11. Ruipeng Gao, Yang Tian, Fan Ye, Guojie Luo, Kaigui Bian, Yizhou Wang, Tao Wang, and Xiaoming Li. 2016. Sextant: Towards ubiquitous indoor localization service by photo-taking of the environment. IEEE Transactions on Mobile Computing 15, 2 (2016), 460--474.
12. Ruipeng Gao, Mingmin Zhao, Tao Ye, Fan Ye, Yizhou Wang, Kaigui Bian, Tao Wang, and Xiaoming Li. 2014. Jigsaw: Indoor floor plan reconstruction via mobile crowdsensing. In Proceedings of the 20th annual international conference on Mobile computing and networking. ACM, 249--260.
13. Ross Girshick. 2015. Fast r-cnn. In Proceedings of the IEEE international conference on computer vision. 1440--1448.
14. John A Hartigan and Manchek A Wong. 1979. Algorithm AS 136: A k-means clustering algorithm. Journal of the Royal Statistical Society. Series C (Applied Statistics) 28, 1 (1979), 100--108.
15. Puneet Jain, Justin Manweiler, and Romit Roy Choudhury. 2015. Overlay: Practical mobile augmented reality. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 331--344.
16. Yifei Jiang, Xin Pan, Kun Li, Qin Lv, Robert P Dick, Michael Hannigan, and Li Shang. 2012. Ariel: Automatic wi-fi based room fingerprinting for indoor localization. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing. ACM, 441--450.
17. Manikanta Kotaru, Kiran Joshi, Dinesh Bharadia, and Sachin Katti. 2015. Spotfi: Decimeter level localization using wifi. In ACM SIGCOMM Computer Communication Review, Vol. 45. ACM, 269--282.
18. Ye-Sheng Kuo, Pat Pannuto, Ko-Jen Hsiao, and Prabal Dutta. 2014. Luxapose: Indoor positioning with mobile phones and visible light. In Proceedings of the 20th annual international conference on Mobile computing and networking. ACM, 447--458.
19. David C Lee, Martial Hebert, and Takeo Kanade. 2009. Geometric reasoning for single image structure recovery. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on. IEEE, 2136--2143.
20. Jeong-Kyun Lee and Kuk-Jin Yoon. 2015. Real-time joint estimation of camera orientation and vanishing points. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1866--1874.
21. Fan Li, Chunshui Zhao, Guanzhong Ding, Jian Gong, Chenxing Liu, and Feng Zhao. 2012. A reliable and accurate indoor localization method using phone inertial sensors. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing. ACM, 421--430.
22. Liqun Li, Pan Hu, Chunyi Peng, Guobin Shen, and Feng Zhao. 2014. Epsilon: A Visible Light Based Positioning System.. In NSDI. 331--343.
23. Guido Ligthart and Frans CA Groen. 1982. A comparison of different autofocus algorithms. In Proc. Sixth International Conference on Pattern Recognition. 597--600.
24. Jorge Lobo and Jorge Dias. 2003. Vision and inertial sensor cooperation using gravity as a vertical reference. IEEE Transactions on Pattern Analysis and Machine Intelligence 25, 12 (2003), 1597--1608.
25. David G Lowe. 1999. Object recognition from local scale-invariant features. In Computer vision, 1999. The proceedings of the seventh IEEE international conference on, Vol. 2. Ieee, 1150--1157.
26. Rainer Mautz and Sebastian Tilch. 2011. Survey of optical indoor positioning systems. In Indoor Positioning and Indoor Navigation (IPIN), 2011 International Conference on. IEEE, 1--7.
27. Faraz M Mirzaei and Stergios I Roumeliotis. 2011. Optimal estimation of vanishing points in a manhattan world. In Computer Vision (ICCV), 2011 IEEE International Conference on. IEEE, 2454--2461.
28. Ana Cris Murillo, J Košecká, Jose Jesus Guerrero, and Carlos Sagüés. 2008. Visual door detection integrating appearance and shape cues. Robotics and Autonomous Systems 56, 6 (2008), 512--521.
29. Anshul Rai, Krishna Kant Chintalapudi, Venkata N Padmanabhan, and Rijurekha Sen. 2012. Zee: Zero-effort crowdsourcing for indoor localization. In Proceedings of the 18th annual international conference on Mobile computing and networking. ACM, 293--304.
30. Davide Scaramuzza. 2011. 1-point-ransac structure from motion for vehicle-mounted cameras by exploiting non-holonomic constraints. International journal of computer vision 95, 1 (2011), 74.
31. Zheng Sun, Shijia Pan, Yu-Chi Su, and Pei Zhang. 2013. Headio: zero-configured heading acquisition for indoor mobile devices through multimodal context sensing. In Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing. ACM, 33--42.
32. Deepak Vasisht, Swarun Kumar, and Dina Katabi. 2016. Decimeter-level localization with a single wifi access point. In 13th USENIX Symposium on Networked Systems Design and Implementation (NSDI 16). USENIX Association, 165--178.
33. He Wang, Souvik Sen, Ahmed Elgohary, Moustafa Farid, Moustafa Youssef, and Romit Roy Choudhury. 2012. No need to war-drive: unsupervised indoor localization. In Proceedings of the 10th international conference on Mobile systems, applications, and services. ACM, 197--210.
34. Horst Wildenauer and Markus Vincze. 2007. Vanishing point detection in complex man-made worlds. In Image Analysis and Processing, 2007. ICIAP 2007. 14th International Conference on. IEEE, 615--622.
35. Hongwei Xie, Tao Gu, Xianping Tao, Haibo Ye, and Jian Lv. 2014. MaLoc: A practical magnetic fingerprinting approach to indoor localization using smartphones. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 243--253.
36. Han Xu, Zheng Yang, Zimu Zhou, Longfei Shangguan, Ke Yi, and Yunhao Liu. 2016. Indoor localization via multi-modal sensing on smartphones. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 208--219.
37. Qiang Xu, Rong Zheng, and Steve Hranilovic. 2015. Idyll: indoor localization using inertial and light sensors on smartphones. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 307--318.
38. Xiaodong Yang and Yingli Tian. 2010. Robust door detection in unfamiliar environments by combining edge and corner features. In Computer Vision and Pattern Recognition Workshops (CVPRW), 2010 IEEE Computer Society Conference on. IEEE, 57--64.
39. Zheng Yang, Chenshu Wu, Zimu Zhou, Xinglin Zhang, Xu Wang, and Yunhao Liu. 2015. Mobility increases localizability: A survey on wireless indoor localization using inertial sensors. ACM Computing Surveys (CSUR) 47, 3 (2015), 54.
40. Haibo Ye, Tao Gu, Xianping Tao, and Jian Lu. 2014. F-Loc: Floor localization via crowdsourcing. In Parallel and Distributed Systems (ICPADS), 2014 20th IEEE International Conference on. IEEE, 47--54.
41. Haibo Ye, Tao Gu, Xiaorui Zhu, Jinwei Xu, Xianping Tao, Jian Lu, and Ning Jin. 2012. FTrack: Infrastructure-free floor localization via mobile phone sensing. In Pervasive Computing and Communications (PerCom), 2012 IEEE International Conference on. IEEE, 2--10.
42. Pengfei Zhou, Mo Li, and Guobin Shen. 2014. Use it free: Instantly knowing your phone attitude. In Proceedings of the 20th annual international conference on Mobile computing and networking. ACM, 605--616.

Index Terms

• Human-centered computing

  • Ubiquitous and mobile computing