Collaborative Annotation of Videos Relying on Weak Consistency

Abstract

This work discusses a distributed interactive video system that supports video annotation using simultaneous hyperlinking by multiple users. The users mark and annotate objects within the video with links to other media such as text, images, websites, or other videos. Annotations are visualized on the client user interface as an overlay close to the objects. Our system is intuitive to use; for example, it contains automatic object-tracking functionality that correctly positions the annotations, even when the form or location of an object changes. Thus, our first contribution discusses the adaptive object-tracking algorithm used for this repositioning. It shows improved precision and reliability in comparison to nonadaptive algorithms. A second key issue is to keep the system responsive when the number of concurrent annotators increases. Thus, we rely on the concept of eventual consistency between different network entities. While this weak form of consistency allows temporary inconsistencies, it ensures that a consistent state can be reached. Thus, the second contribution is the design and evaluation of our distributed interactive video system, which relies on the weak consistency paradigm.

References

1. Ismail Ari, Bo Hong, Ethan Miller, Scott Brandt, and Darrell Long. 2003. Managing flash crowds on the internet. In IEEE Analysis and Simulation of Computer Telecommunications Systems.
2. Peter Bailis and Ali Ghodsi. 2013. Eventual consistency today: Limitations, extensions, and beyond. ACM Queue - Storage 11, 3 (March 2013).
3. Herbert Bay and Tinne Tuytelaars. 2006. SURF: Speeded up robust features. In European Conference on Computer Vision.
4. David Bermbach and Stefan Tai. 2011. Eventual consistency: How soon is eventual? An evaluation of Amazon S3’s consistency behavior. In ACM Workshop on Middleware for Service Oriented Comp.
5. Ahmed Bouajjani, Constantin Enea, and Jad Hamza. 2014. Verifying eventual consistency of optimistic replication systems. ACM SIGPLAN Notices 49, 1 (Jan. 2014).
6. Nicolas Bouillot and Eric Gressier-Soudan. 2004. Consistency models for distributed interactive multimedia applications. ACM SIGOPS Operating Systems Review 38, Article 4 (Oct. 2004).
7. Eric Brewer. 2000. Towards robust distributed systems. In ACM Symposium on Principles of Distributed Computing.
8. Dick Bulterman. 2004. Creating peer-level video annotations for web-based multimedia. (2004).
9. Sebastian Burckhardt, Manuel Fähndrich, Daan Leijen, and Benjamin P. Wood. 2012. Cloud types for eventual consistency. In ECOOP 2012 Object-Oriented Programming, James Noble (Ed.). Lecture Notes in Computer Science, Vol. 7313. Springer, Berlin, 283--307.
10. Pablo Cesar, Dick C. A. Bulterman, David Geerts, Jack Jansen, Hendrik Knoche, and William Seager. 2008. Enhancing social sharing of videos: Fragment, annotate, enrich, and share. In ACM International Conference on Multimedia.
11. Pei-Yu Chi and Henry Lieberman. 2011. Raconteur: Integrating authored and real-time social media. In ACM Conference on Human Factors in Computing Systems.
12. Dorin Comaniciu, Visvanathan Ramesh, and Peter Meer. 2003. Kernel-based object tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence 25, 5 (2003), 564--577.
13. Andreas Girgensohn, Frank Shipman, and Lynn Wilcox. 2003. Hyper-Hitchcock: Towards the easy authoring of interactive video. In Human-Computer Interaction.
14. Dan Goldman, Chris Gonterman, Brian Curless, David Salesin, and Steven Seitz. 2008. Video object annotation, navigation, and composition. In ACM Symposium on UI Software and Technology.
15. Tovi Grossman and George Fitzmaurice. 2010. ToolClips: Investigation of contextual video assistance for functionality understanding. In ACM Conference on Human Factors in Computing Systems.
16. Sanny Gustavsson and Sten F. Andler. 2002. Self-stabilization and eventual consistency in replicated real-time databases. In ACM Workshop on Self-Healing Systems.
17. Oskar Juhlin, Erika Reponen, Frank Bentley, and David Kirk. 2011. Video interaction—Making broadcasting a successful social media. In ACM Conference on Human Factors in Computing Systems.
18. Stephan Kopf, Stefan Wilk, and Wolfgang Effelsberg. 2012. Bringing videos to social media. In IEEE International Conference on Multimedia and Expo.
19. Rodrigo Laiola Guimarães, Pablo Cesar, and Dick Bulterman. 2010. Creating and sharing personalized time-based annotations of videos on the web. In ACM Symposium on Document Engineering.
20. Rodrigo Laiola Guimarães, Pablo Cesar, and Dick C. A. Bulterman. 2012. “Let me comment on your video”. In ACM Brazilian Symposium on Multimedia and the Web.
21. Xi Li, Weiming Hu, Chunhua Shen, Zhongfei Zhang, Anthony Dick, and Anton Van Den Hengel. 2013. A survey of appearance models in visual object tracking. ACM Transactions on Intelligent Systems and Technology 4, 4 (Sept. 2013), 1--48.
22. Bruce D. Lucas and Takeo Kanade. 1981. An iterative image registration technique with an application to stereo vision. In ACM International Joint Conference on Artificial Intelligence.
23. Jos M. Martinez, Rob Koenen, and Fernando Pereira. 2002. MPEG-7: The generic multimedia content description standard. IEEE Multimedia 9, 2 (2002), 78--87.
24. Carl D. McDaniel and Roger H. Gates. 1998. Marketing Research Essentials (1st ed.). Taylor & Francis.
25. Britta Meixner, Stefan John, and Christian Handschigl. 2015. SIVA suite: Framework for hypervideo creation, playback and management. In ACM Conference on Multimedia.
26. Britta Meixner, Katarzyna Matusik, and Harald Kosch. 2012. Towards an easy to use authoring tool for interactive non-linear video. Multimedia Tools and Applications 70, 2 (2012), 1251--1276.
27. Isabelle Mirbel, Barbara Pernici, Timos Sellis, Savos Tserkezoglou, and Michalis Vazirgiannis. 2000. Checking the temporal integrity of interactive multimedia documents. ACM International Journal on Very Large Data Bases 9, 2 (2000), 111--130.
28. Mu Mu, Steven Simpson, Craig Bojko, Matthew Broadbent, James Brown, Andreas Mauthe, Nicholas Race, and David Hutchison. 2013. Storisphere: From TV watching to community story telling. IEEE Communications Magazine 51, 8 (Aug. 2013), 112--119.
29. Xiangming Mu, Gary Marchionini, and Amy Pattee. 2003. The interactive shared educational environment: User interface, system architecture and field study. In IEEE Conference on Digital Libraries.
30. Marius Muja and David G. Lowe. 2009. Fast approximate nearest neighbours with automatic algorithm configuration. In International Conference on Computer Vision Theory and Application.
31. Joseph M. Hellerstein, Peter Alvaro, Neil Conway, and William R. Marczak. 2011. Consistency analysis in bloom: A CALM and collected approach. In Conference on Innovative Data Systems Research.
32. Muntasir Raihan Rahman, Wojciech Golab, Alvin AuYoung, Kimberly Keeton, and Jay J. Wylie. 2012. Toward a principled framework for benchmarking consistency. In USENIX Workshop on Hot Topics in System Dependability.
33. Yasushi Saito and Marc Shapiro. 2005. Optimistic replication. Computing Surveys 37, 1 (2005), 42--81.
34. Peter Sand and Seth Teller. 2008. Particle video: Long-range motion estimation using point trajectories. International Journal of Computer Vision 80, 1 (May 2008), 72--91.
35. Nitin Sawhney, David Balcom, and Ian Smith. 1996. HyperCafe: Narratic and aesthetic properties of hypervideo. In ACM International Conference on Hypertext.
36. Andrew N. Stein and Martial Hebert. 2009. Local detection of occlusion boundaries in video. Elsevier Journal on Image and Vision Computing 27, 5 (2009), 514--522.
37. Werner Vogels. 2008. Eventually consistent. ACM Queue 6, 6 (Oct. 2008), 14.
38. Shiuh-Ku Weng, Chung-Ming Kuo, and Shu-Kang Tu. 2006. Video object tracking using adaptive Kalman filter. Journal of Visual Communication and Image Representation 17. 6 (Dec. 2006), 1190--1208.
39. Stefan Wilk and Wolfgang Effelsberg. 2014. The influence of camera shakes, harmful occlusions and camera misalignment on the perceived quality in user generated video. In IEEE International Conference on Multimedia and Expo. 1--6.
40. Stefan Wilk, Stephan Kopf, and Wolfgang Effelsberg. 2012. Robust tracking for interactive social video. In IEEE Workshop on Applications of Computer Vision. 105--110.
41. Stefan Wilk, Stephan Kopf, and Wolfgang Effelsberg. 2013. Social video: A collaborative video annotation environment to support e-learning . In AACE Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications (EdMedia).
42. Alper Yilmaz, Omar Javed, and Mubarak Shah. 2006. Object tracking: A survey. Computing Surveys 38, 4 (Dec. 2006), 1--43.

Index Terms

• Computer systems organization

  • Architectures

    • Distributed architectures

      • Client-server architectures

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction paradigms

      • Hypertext / hypermedia