Tingxin Yan
ABSTRACT
Mobile phones are becoming increasingly sophisticated with a rich set of on-board sensors and ubiquitous wireless connectivity. However, the ability to fully exploit the sensing capabilities on mobile phones is stymied by limitations in multimedia processing techniques. For example, search using cellphone images often encounters high error rate due to low image quality.
In this paper, we present CrowdSearch, an accurate image search system for mobile phones. CrowdSearch combines automated image search with real-time human validation of search results. Automated image search is performed using a combination of local processing on mobile phones and backend processing on remote servers. Human validation is performed using Amazon Mechanical Turk, where tens of thousands of people are actively working on simple tasks for monetary rewards. Image search with human validation presents a complex set of tradeoffs involving energy, delay, accuracy, and monetary cost. CrowdSearch addresses these challenges using a novel predictive algorithm that determines which results need to be validated, and when and how to validate them. CrowdSearch is implemented on Apple iPhones and Linux servers. We show that CrowdSearch achieves over 95% precision across multiple image categories, provides responses within minutes, and costs only a few cents.
- M. Azizyan, I. Constandache, and R. Choudhury. Surroundsense: mobile phone localization via ambience fingerprinting. In Proceedings of MobiCom 09, Sep 2009. Google Scholar
Digital Library
- R. Baeza-Yates and B. Ribeiro--Neto. Modern Information Retrieval. ACM Press, 1999. Google ScholarDigital Library
- K. P. Burnham and A. D. R. Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach, Second Edition. Springer Science, New York, 2002.Google Scholar
- A. T. Campbell, S. B. Eisenman, N. D. Lane, E. Miluzzo, and R. A. Peterson. People-centric urban sensing. In WICON'06: Proceedings of the 2nd annual international workshop on Wireless internet, page 18, New York, NY, USA, 2006. ACM. Google ScholarDigital Library
- O. Chum, J. Philbin, M. Isard, and A. Zisserman. Scalable near identical image and shot detection. In Proceedings of CIVR'07, pages 549--556, New York, NY, USA, 2007. Google ScholarDigital Library
- E. Cuervo, A. Balasubramanian, D. ki Cho, A. Wolman, S. Saroiu, R. Chandra, and P. Bahl. Maui: Making smartphones last longer with code offload. In In Proceedings of ACM MobiSys, 2010. Google ScholarDigital Library
- S. B. Eisenman, N. D. Lane, E. Miluzzo, R. A. Peterson, G. seop Ahn, and A. T. Campbell. Metrosense project: People-centric sensing at scale. In In WSW 2006 at Sensys, 2006.Google Scholar
- A. Kittur, E. Chi, and B. Suh. Crowdsourcing user studies with mechanical turk. CHI 2008, Jan 2008. Crowdsourcing applied to user study. Google ScholarDigital Library
- D. G. Lowe. Distinctive image features from scale-invariant keypoints, 2003. Google ScholarDigital Library
- H. Lu, W. Pan, N. D. Lane, T. Choudhury, and A. T. Campbell. Soundsense: scalable sound sensing for people-centric applications on mobile phones. In MobiSys, pages 165--178, 2009. Google ScholarDigital Library
- M.-E. Nilsback. An automatic visual Flora -- segmentation and classification of flowers images. PhD thesis, University of Oxford, 2009.Google Scholar
- M.-E. Nilsback and A. Zisserman. Automated flower classification over a large number of classes. In Proceedings of the Indian Conference on Computer Vision, Graphics and Image Processing, Dec 2008. Google ScholarDigital Library
- J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman. Object retrieval with large vocabularies and fast spatial matching. In CVPR, 2007.Google ScholarCross Ref
- V. S. Sheng, F. Provost, and P. G. Ipeirotis. Get another label? improving data quality and data mining using multiple, noisy labelers. In In Proceeding of KDD'08, pages 614--622, 2008. Google ScholarDigital Library
- A. Sorokin and D. Forsyth. Utility data annotation with amazon mechanical turk. Computer Vision and Pattern Recognition Workshops, Jan 2008.Google ScholarCross Ref
- http://images.google.com/imagelabeler/. Google Labeler.Google Scholar
- https://www.livework.com/. LiveWork: Outsource Business Tasks To Teams of On-Demand Workers.Google Scholar
- http://www.abiresearch.com/research/1002762-US+Mobile+Email+and+Mobile+Web+Access+Trends. US Mobile Email and Mobile Web Access Trends -- 2008.Google Scholar
- http://www.chacha.com/. ChaCha: Real people answering your questions.Google Scholar
- http://www.crowdspirit.com/. CrowdSpirit: enables businesses to involve innovators from outside the company directly in the design of innovative products and services.Google Scholar
- http://www.google.com/mobile/products/search.html#p=default. Goggle: Google image search on mobile phones.Google Scholar
- http://www.sensorplanet.org/. Sensor Planet: a mobile device-centric large-scale Wireless Sensor Networks.Google Scholar
- http://www.taskcn.com/. Taskcn: A platform for outsourcing tasks.Google Scholar
- http://www.theextraordinaries.org/crowdsourcing.html. The Extraordinaries.Google Scholar
- http://www.topcoder.com/. www.topcoder.com.Google Scholar
- http://www.vlfeat.org/ vedaldi/code/siftpp.html. SIFT++: a lightweight C++ implementation of SIFT detector and descriptor.Google Scholar
- http://www.wired.com/gadgetlab/2008/12/amazons-iphone/. Amazon Mobile: Amazon Remember.Google Scholar
- L. von Ahn and L. Dabbish. Labeling images with a computer game. In CHI'04: Proceedings of the SIGCHI conference on Human factors in computing systems, pages 319--326, New York, NY, USA, 2004. ACM Press. Google ScholarDigital Library
- L. von Ahn, B. Maurer, C. Mcmillen, D. Abraham, and M. Blum. recaptcha: Human-based character recognition via web security measures. Science, 321(5895): 1465--1468, August 2008.Google Scholar
- T. Yan, D. Ganesan, and R. Manmatha. Distributed image search in camera sensor networks. In Proceedings of SenSys 2008, Jan 2008. Google ScholarDigital Library
- C. Zhu, K. Li, Q. Lv, L. Shang, and R. Dick. iscope: personalized multi-modality image search for mobile devices. In Proceedings of Mobisys'09, Jun 2009. Google ScholarDigital Library
Index Terms
- CrowdSearch: exploiting crowds for accurate real-time image search on mobile phones
Recommendations
Wally: crowd powered image matching on tablets
CrowdKDD '12: Proceedings of the First International Workshop on Crowdsourcing and Data MiningIn this paper we propose a crowd sourced approach for solving large scale object retrieval. We have built a tablet application which displays a query image and a database image. The crowd provides their input to indicate, if there is a match between the ...
Design Strategies for Efficient Access to Mobile Device Users via Amazon Mechanical Turk
CrowdSenSys '17: Proceedings of the First ACM Workshop on Mobile Crowdsensing Systems and ApplicationsIt is often challenging to access a pool of mobile device users and instruct them to perform an interactive task. Yet such data is often vital to provide design insight at various stages of the design process of a mobile application, service or system. ...
Modus Operandi of Crowd Workers: The Invisible Role of Microtask Work Environments
The ubiquity of the Internet and the widespread proliferation of electronic devices has resulted in flourishing microtask crowdsourcing marketplaces, such as Amazon MTurk. An aspect that has remained largely invisible in microtask crowdsourcing is that ...
Comments