skip to main content
10.1145/2526667.2526671acmotherconferencesArticle/Chapter ViewAbstractPublication PagessensecamConference Proceedingsconference-collections
research-article

Do you see what I see: crowdsource annotation of captured scenes

Authors Info & Claims
Published:18 November 2013Publication History

ABSTRACT

The Archive of Many Outdoor Scenes has captured 400 million images. Many of these cameras and images are of street intersections, a subset of which has experienced built environment improvements during the past seven years. We identified six cameras in Washington, DC, and uploaded 120 images from each before a built environment change (2007) and after (2010) to the crowdsourcing website Amazon Mechanical Turk (n=1,440). Five unique MTurk workers annotated each image, counting the number of pedestrians, cyclists, and vehicles. Two trained Research Assistants completed the same tasks. Reliability and validity statistics of MTurk workers revealed substantial agreement in annotating captured images of pedestrians and vehicles. Using the mean annotation of four MTurk workers proved most parsimonious for valid results. Crowdsourcing was shown to be a reliable and valid workforce for annotating images of outdoor human behavior.

References

  1. Jacobs, N., et al., The Global Network of Outdoor Webcams: Properties and Applications. ACM International Conference on Advances in Geographic Information Systems (SIGSPATIAL GIS), 2009: p. 111--120. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Hipp, J. A., et al., Emerging Technologies: Webcams and Crowd-Sourcing to Identify Active Transportation. American Journal of Preventive Medicine, 2013. 44(1): p. 96--97.Google ScholarGoogle Scholar
  3. Landis, J. R. and G. G. Koch, The Measurement of Observer Agreement for Categorical Data. Biometrics, 1977. 33(1): p. 159--174.Google ScholarGoogle Scholar

Index Terms

  1. Do you see what I see: crowdsource annotation of captured scenes

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in
          • Published in

            cover image ACM Other conferences
            SenseCam '13: Proceedings of the 4th International SenseCam & Pervasive Imaging Conference
            November 2013
            99 pages
            ISBN:9781450322478
            DOI:10.1145/2526667

            Copyright © 2013 Owner/Author

            Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 18 November 2013

            Check for updates

            Qualifiers

            • research-article

            Acceptance Rates

            SenseCam '13 Paper Acceptance Rate9of22submissions,41%Overall Acceptance Rate9of22submissions,41%

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader