Mobile image recognition: architectures and tradeoffs

ABSTRACT

We argue that the most desirable architecture for mobile image recognition runs the complete algorithm on the mobile device. Alternative solutions that run the recognizer on a remote server will not be as desirable because of the delay between image capture and receipt of a result that can cause users to abandon the technique. We present a method for mobile recognition of paper documents and an application to newspapers that lets readers retrieve electronic data linked to articles, photos, and advertisements. We show that the index for a reasonable collection of daily newspapers can be downloaded in less than a minute and will fit in the memory of today's mid-range smart phones. Experimental results show that the recognition system has an overall error rate of less than 1%. We achieved a run time of 1.2 secs. per image with a collection of 140 newspaper pages on an HTC-8282 Windows Mobile phone.

References

1. Commercial offerings that recognize individual images submitted from camera phones include www.doog.mobi and www.snaptell.com.
2. J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman, "Object retrieval with large vocabularies and fast spatial matching," Proc. of the IEEE CVPR, 2007.
3. J. J. Hull, B. Erol, J. Graham, Q. Ke, H. Kishi, J. Moraleda, and D. Van Olst, "Paper-Based Augmented Reality," 17th Int. Conf. on Augmented Reality and Telexistence, Esbjerg, Denmark, Nov. 28--30 2007, 205--209.
4. B. Erol, E. Antunez, and J. J. Hull, "HOTPAPER: multimedia interaction with paper using mobile phones," Proc. of the 16th ACM Intl. Conf. on Multimedia, Vancouver, Canada, 2008, pp. 399--408.
5. S. Dekleva, J. P. Shim, U. Varshney, and G. Knoerzer, "Evolution and emerging issues in mobile wireless networks," Comm. ACM, v. 50, no. 6, June 2007, pp. 38--43.
6. T. Nakai, K. Kise, and M. Iwamura, "Use of affine invariants in locally likely arrangement hashing for camera-based document image retrieval," Lecture Notes in Computer Science (7th International Workshop DAS2006, vol. 3872, 2006.
7. X. Liu and D. Doermann, "Mobile Retriever: access to digital documents from their physical source," International Journal on Document Analysis and Recognition, vol. 11, 2008, pp. 19--27.
8. D. Wagner, G. Reitmayr, A. Mulloni, T. Drummond, and D. Schmalstieg, "Pose tracking from natural features on mobile phones," Proc. of the 7th IEEE/ACM Int. Symp. on Mixed and Augmented Reality (Sept. 15--18, 2008), pp. 125--134.
9. G. Takacs, V. Chandrasekhar, N. Gelfand, Y. Xiong, W. C. Chen, T. Bismpigiannis, R. Grzeszczuk, K. Pulli, and B. Girod, "Outdoors augmented reality on mobile phone using loxel-based visual feature organization," ACM International Conference on Multimedia Information Retrieval (MIR'08), Vancouver, Canada, Oct. 2008.

Index Terms

• Applied computing

  • Document management and text processing

• Human-centered computing

  • Human computer interaction (HCI)