SweepLoc: Automatic Video-based Indoor Localization by Camera Sweeping

Abstract

Indoor localization based on visual landmarks has received much attention in commercial sites with rich features (e.g., shopping malls, museums) recently because landmarks are relatively stable over a long time. Prior arts often require a user to take multiple independent images around his/her location, and manually confirm shortlisted landmarks. The process is sophisticated, inconvenient, slow, unnatural and error-prone. To overcome these limitations, we propose SweepLoc, a novel, efficient and automatic video-based indoor localization system. SweepLoc mimics our natural scanning around to identify nearby landmarks in an unfamiliar site to localize.

In SweepLoc, a user simply takes a short video clip (about 6 to 8 seconds) of his/her surroundings by sweeping the camera. Using correlation and scene continuity between successive video frames, it automatically and efficiently selects key frames (where potential landmarks are centered) and subsequently reduces the decision error on landmarks. With identified landmarks, SweepLoc formulates an optimization problem to locate the user, taking compass noise and floor map constraint into account. We have implemented SweepLoc in Android platform. Our extensive experimental results in a food plaza and a premium mall demonstrate that SweepLoc is fast (less than 1 second to localize), and achieves substantially better accuracy as compared with the state-of-the-art approaches (reducing the localization error by 30%).

References

1. P. Bahl and V. N. Padmanabhan. 2000. RADAR: an in-building RF-based user location and tracking system. In Proc. of IEEE INFOCOM, Vol. 2. IEEE, 775--784 vol.2.
2. Herbert Bay, Tinne Tuytelaars, and Luc Van Gool. 2006. SURF: speeded up robust features. In Proc. of Springer ECCV. Springer, Berlin, Heidelberg, 404--417.
3. Jiang Dong, Yu Xiao, Marius Noreikis, Zhonghong Ou, and Antti Ylä-Jääski. 2015. iMoon: using smartphones for image-based indoor navigation. In Proc. of ACM SenSys. New York, NY, USA, 85--97.
4. Jiang Dong, Yu Xiao, Zhonghong Ou, Yong Cui, and Antti Ylä-Jääski. 2016. Indoor Tracking Using Crowdsourced Maps. In Proc. of IEEE IPSN. Article 5, 6 pages.
5. G David Forney. 1973. The viterbi algorithm. Proc. IEEE 61, 3 (1973), 268--278.
6. R. Gao, Y. Tian, F. Ye, G. Luo, K. Bian, Y. Wang, T. Wang, and X. Li. 2016. Sextant: Towards ubiquitous indoor localization service by photo-taking of the environment. IEEE Trans. Mob. Comput. 15, 2 (Feb 2016), 460--474.
7. R. Gao, B. Zhou, F. Ye, and Y. Wang. 2017. Knitter: Fast, resilient single-user indoor floor plan construction. In Proc. IEEE INFOCOM. IEEE, 1--9.
8. Cole Gleason, Dragan Ahmetovic, Saiph Savage, Carlos Toxtli, Carl Posthuma, Chieko Asakawa, Kris M. Kitani, and Jeffrey P. Bigham. 2018. Crowdsourcing the Installation and Maintenance of Indoor Localization Infrastructure to Support Blind Navigation. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2, 1, Article 9 (March 2018), 25 pages.
9. Fei Gu, Jianwei Niu, and Lingjie Duan. 2017. WAIPO: A Fusion-Based Collaborative Indoor Localization System on Smartphones. IEEE/ACM Trans. Netw. 25, 4 (Aug 2017), 2267--2280.
10. B. Guo, Q. Han, H. Chen, L. Shangguan, Z. Zhou, and Z. Yu. 2017. The Emergence of Visual Crowdsensing: Challenges and Opportunities. Commun. Surveys Tuts. 19, 4 (Fourthquarter 2017), 2526--2543.
11. J. F. Henriques, R. Caseiro, P. Martins, and J. Batista. 2015. High-speed tracking with kernelized correlation filters. IEEE Trans. Pattern Anal. Mach. Intell. 37, 3 (March 2015), 583--596.
12. Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. 2014. Caffe: convolutional architecture for fast feature embedding. In Proc. of ACM MM. 675--678.
13. Hernisa Kacorri, Eshed Ohn-Bar, Kris M. Kitani, and Chieko Asakawa. 2018. Environmental Factors in Indoor Navigation Based on Real-World Trajectories of Blind Users. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, Article 56, 12 pages.
14. Alex Kendall, Matthew Grimes, and Roberto Cipolla. 2015. PoseNet: a convolutional network for real-time 6-DOF camera relocalization. In Proc. of IEEE ICCV. 2938--2946.
15. T. H. Kim, S. Nah, and K. M. Lee. 2017. Dynamic Video Deblurring using a Locally Adaptive Linear Blur Model. IEEE Trans. Pattern Anal. Mach. Intell. PP, 99 (Oct 2017), 1--1.
16. Patrick Lazik, Niranjini Rajagopal, Oliver Shih, Bruno Sinopoli, and Anthony Rowe. 2015. ALPS: A Bluetooth and Ultrasound Platform for Mapping and Localization. In Proc. ACM Sensys. ACM, 73--84.
17. Kai Liu, Hao Zhang, Joseph Kee-Yin Ng, Yusheng Xia, Liang Feng, Victor CS Lee, and Sang H Son. 2018. Toward Low-Overhead Fingerprint-Based Indoor Localization via Transfer Learning: Design, Implementation, and Evaluation. IEEE Trans. Ind. Informat. 14, 3 (2018), 898--908.
18. Z. Liu, L. Zhang, Q. Liu, Y. Yin, L. Cheng, and R. Zimmermann. 2017. Fusion of magnetic and visual sensors for indoor localization: infrastructure-free and more effective. IEEE Trans. Multimedia 19, 4 (April 2017), 874--888.
19. Chengwen Luo, Hande Hong, Mun Choon Chan, Jianqiang Li, Xinglin Zhang, and Zhong Ming. 2018. MPiLoc: Self-Calibrating Multi-Floor Indoor Localization Exploiting Participatory Sensing. IEEE Trans. Mob. Comput. 17, 1 (2018), 141--154.
20. Raul Mur-Artal and Juan D Tardós. 2017. Orb-slam2: An opensource slam system for monocular, stereo, and rgb-d cameras. IEEE Trans. Robot. 33, 5 (2017), 1255--1262.
21. S. Papaioannou, A. Markham, and N. Trigoni. 2017. Tracking People in Highly Dynamic Industrial Environments. IEEE Trans. Mob. Comput. 16, 8 (Aug 2017), 2351--2365.
22. Valter Pasku, Alessio De Angelis, Darmindra D. Arumugam, Marco Dionigi, Paolo Carbone, Antonio Moschitta, and David S. Ricketts. 2017. Magnetic Field-Based Positioning Systems. IEEE Commun. Surveys Tuts. 19, 3 (Mar 2017), 2003--2017.
23. Claudio Piciarelli. 2016. Visual indoor localization in known environments. IEEE Signal Process. Lett. 23, 10 (2016), 1330--1334.
24. Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster R-CNN: towards real-time object detection with region proposal networks. In Proc. of NIPS. MIT Press, 91--99.
25. Ethan Rublee, Vincent Rabaud, Kurt Konolige, and Gary Bradski. 2011. ORB: an efficient alternative to SIFT or SURF. In Proc. of IEEE ICCV. 2564--2571.
26. Torsten Sattler, Bastian Leibe, and Leif Kobbelt. 2016. Exploiting Spatial and Co-visibility Relations for Image-Based Localization. Springer International Publishing, Cham, 165--187.
27. Karen Simonyan and Andrew Zisserman. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. International Conference on Learning Representations (2015).
28. Masato Sugasaki and Masamichi Shimosaka. 2017. Robust Indoor Localization Across Smartphone Models with Ellipsoid Features from Multiple RSSIs. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 3, Article 103 (Sept. 2017), 16 pages.
29. Xiaohua Tian, Mei Wang, Wenxin Li, Binyao Jiang, Dong Xu, Xinbing Wang, and Jun Xu. 2018. Improve Accuracy of Fingerprinting Localization with Temporal Correlation of the RSS. IEEE Trans. Mob. Comput. 17, 1 (2018), 113--126.
30. Matthew Wall. 1996. GAlib: A C++ library of genetic algorithm components. Mechanical Engineering Department, Massachusetts Institute of Technology 87 (1996), 54.
31. S. Wang, S. Fidler, and R. Urtasun. 2015. Lost Shopping! Monocular Localization in Large Indoor Spaces. In Proc. IEEE ICCV. IEEE, 2695--2703.
32. Yu-Lin Wei, Chang-Jung Huang, Hsin-Mu Tsai, and Kate Ching-Ju Lin. 2017. CELLI: Indoor Positioning Using Polarized Sweeping Light Beams. In Proc. of ACM MobiSys. ACM, 136--147.
33. Hongkai Wen, Sen Wang, Ronnie Clark, Savvas Papaioannou, and Niki Trigoni. 2016. Poster: Efficient Visual Positioning with Adaptive Parameter Learning. In Proc. of IEEE IPSN. IEEE Press, Piscataway, NJ, USA, Article 34, 2 pages.
34. M. Werner, M. Kessel, and C. Marouane. 2011. Indoor positioning using smartphone camera. In Proc. of IEEE IPIN. 1--6.
35. Chenshu Wu, Zheng Yang, and Chaowei Xiao. 2018. Automatic Radio Map Adaptation for Indoor Localization Using Smartphones. IEEE Trans. Mob. Comput. 17, 3 (2018), 517--528.
36. K. Wu, Jiang Xiao, Youwen Yi, Min Gao, and L. M. Ni. 2012. FILA: Fine-grained indoor localization. In Proc. IEEE INFOCOM. IEEE, 2210--2218.
37. Han Xu, Zheng Yang, Zimu Zhou, and Chunyi Peng Ke Yi. 2017. TUM: Towards Ubiquitous Multi-Device Localization for Cross-Device Interaction. In Proc. IEEE INFOCOM.
38. Han Xu, Zheng Yang, Zimu Zhou, Longfei Shangguan, Ke Yi, and Yunhao Liu. 2015. Enhancing wifi-based localization with visual clues. In Proc. of ACM UbiComp. ACM, New York, New York, USA, 963--974.
39. Han Xu, Zheng Yang, Zimu Zhou, Longfei Shangguan, Ke Yi, and Yunhao Liu. 2016. Indoor Localization via Multimodal Sensing on Smartphones. In Proc. ACM UbiComp. ACM, 208--219.
40. Yi Yao, Besma Abidi, Narjes Doggaz, and Mongi Abidi. 2006. Evaluation of sharpness measures and search algorithms for the auto focusing of high-magnification images. In Visual Information Processing XV, Vol. 6246. International Society for Optics and Photonics, 62460G.
41. A. Yassin, Y. Nasser, M. Awad, A. Al-Dubai, R. Liu, C. Yuen, R. Raulefs, and E. Aboutanios. 2017. Recent Advances in Indoor Localization: A Survey on Theoretical Approaches and Applications. Commun. Surveys Tuts. 19, 2 (Secondquarter 2017), 1327--1346.
42. Xuehan Ye, Yongcai Wang, Yuhe Guo, Wei Hu, and Deying Li. 2018. Accurate and Efficient Indoor Location by Dynamic Warping in Sequence-Type Radio-Map. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2, 1 (2018), 50.
43. Moustafa Youssef and Ashok Agrawala. 2005. The horus WLAN location determination system. In Proc. of ACM MobiSys. 205--218.
44. Matthew D. Zeiler and Rob Fergus. 2014. Visualizing and Understanding Convolutional Networks. In Proc. of Springer ECCV. Springer, Cham, 818--833.
45. Hongwei Zhang, Barry Lennox, Peter R Goulding, and Andrew YT Leung. 2000. A float-encoded genetic algorithm technique for integrated optimization of piezoelectric actuator and sensor placement and feedback gains. Smart Materials and Structures 9, 4 (2000), 552.
46. X. Zhang, A. K. S. Wong, C. T. Lea, and R. S. K. Cheng. 2018. Unambiguous Association of Crowd-Sourced Radio Maps to Floor Plans for Indoor Localization. IEEE Trans. Mob. Comput. 17, 2 (Feb 2018), 488--502.
47. Yongtuo Zhang, Wen Hu, Weitao Xu, Hongkai Wen, and Chun Tung Chou. 2016. NaviGlass: indoor localisation using smart glasses. In Proc. of ACM EWSN. 205--216.
48. Y. Zheng, G. Shen, L. Li, C. Zhao, M. Li, and F. Zhao. 2017. Travi-Navi: Self-Deployable Indoor Navigation System. IEEE/ACM Trans. Netw. 25, 5 (Oct 2017), 2655--2669.
49. Shilin Zhu and Xinyu Zhang. 2017. Enabling High-Precision Visible Light Localization in Today's Buildings. In Proc. of ACM MobiSys. ACM, 96--108.
50. Jinbo Zuo, Shuo Liu, Hao Xia, and Yanyou Qiao. 2018. Multi-Phase Fingerprint Map Based on Interpolation for Indoor Localization Using iBeacons. IEEE Sensors J. (2018), 3351--3359.

Index Terms

• Human-centered computing

  • Ubiquitous and mobile computing

    • Ubiquitous and mobile computing systems and tools

• Networks

  • Network services

    • Location based services