Theodoros Papatheodorou
Abstract
Conventionally, spectators enjoy films passively. This paper describes an interactive film installation that invites participants to use their entire body as a query to search and explore a large corpus of musical films in a novel way. Using an RGB camera, ML-based skeleton tracking technology and a unique pose and film indexing system, this installation tracks a participant's movements and mirrors them in real-time by finding matching poses among hundreds of thousands from well-known musicals. When the participant freezes in a pose, the installation instantly plays back the short film clip that begins with that pose, immersing them in the music and dance from musicals of different eras. This approach explores themes of tangible interfaces and the new possibilities that emerge from employing embodied interaction to traverse the dance pose space, which is traditionally difficult to index and interact with in real time. The pose indexing system and whole-body interaction we propose in this paper open new pathways for cultural participation, as they lend themselves to different datasets and require no technical skills from participants.
- Alissa N. Antle, Allen Bevans, Theresa Jean Tanenbaum, Katie Seaborn and Sijie Wang. 2010. Futura: design for collaborative learning and game play on a multi-touch digital tabletop. In Proceedings of the fifth international conference on Tangible, embedded and embodied interaction (TEI '11). Association for Computing Machinery, New York, NY, USA, 93--100.Google Scholar
Digital Library
- Pina Bausch. n.d.. Official website, Retrieved April 11, 2023 from https://www.pinabausch.org/Google Scholar
- Siobhan Davies et. al. 2008. RePlay. Retrieved January 17, 2023 from http://www.siobhandaviesreplay.com/Google Scholar
- Exceptions to copyright. n.d.. UK government website, Retrieved April 11, 2023 from https://www.gov.uk/guidance/exceptions-to-copyrightGoogle Scholar
- Jane Friedhoff and Irene Alvarado. 2018. Move Mirror. Retrieved January 17, 2023 from https://experiments.withgoogle.com/move-mirrorGoogle Scholar
- Benedikt Gross. 2016. Alphabet From The Sky. Penguin Random HouseGoogle Scholar
- Zach Lieberman. 2016. Land Lines. Retrieved January 17, 2023 from https://experiments.withgoogle.com/land-linesGoogle Scholar
- Camillo Lugaresi, Jiuqiang Tang, Hadon Nash, Chris McClanahan, Esha Uboweja, Michael Hays, Fan Zhang, Chuo-Ling Chang, Ming Yong, Juhyun Lee, Wan-Teh Chang, Wei Hua, Manfred Georg, Matthias Grundmann. 2019. MediaPipe: A Framework for Perceiving and Processing Reality.Third Workshop on Computer Vision for AR/VR at IEEE Computer Vision and Pattern RecognitionGoogle Scholar
- Leilah Lyons, Brian Slattery, Priscilla Jimenez, Brenda Lopez and Tom Moher. 2012. Don't forget about the sweat: effortful embodied interaction in support of learning. In Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied InteractionGoogle ScholarDigital Library
- Christian Marclay. 2010. The Clock. Retrieved January 17, 2023 from https://www.tate.org.uk/art/artworks/marclay-the-clock-t14038Google Scholar
- Memo Akten. 2009. Body Paint, Retrieved April 11, 2023 from https://www.memo.tv/works/bodypaint/Google Scholar
- S Price, Y Rogers, M Scaife, D Stanton, H Neale. 2003. Using 'tangibles' to promote novel forms of playful learning, Interacting with Computers, Volume 15, Issue 2, Pages 169--185,Google ScholarCross Ref
- Jennifer G. Sheridan, Nick Bryan-Kinns. 2008. Designing for performative tangible interaction. Int. J. Arts and Technology, Vol. 1, Nos. 3/4Google ScholarCross Ref
- Tsuchida, Shuhei & Fukayama, Satoru & Goto, Masataka. 2019. Query-by-Dancing: A Dance Music Retrieval System Based on Body-Motion Similarity. 10.1007/978-3-030-05710-7_21Google Scholar
- Gene Youngblood. 1970. Expanded Cinema. P. Dutton & Co., Inc.Google Scholar
- Oren Zuckerman, Ayelet Gal-Oz. 2013. To TUI or not to TUI: Evaluating performance and preference in tangible vs. graphical user interfaces. Internat. J. Human-Computer Studies, Vol 71, 803--820Google ScholarDigital Library
Index Terms
- Lights! Dance! Freeze!: Exploring the dance-musical filmic space using embodied search in an interactive installation
Recommendations
Lights! Dance! Freeze!: An interactive installation for exploring dance-musicals using embodied search
VINCI '23: Proceedings of the 16th International Symposium on Visual Information Communication and InteractionTraditionally, movie watchers absorb content passively. This paper describes an immersive film installation where participants use their whole body to browse and interact with a large database of musical films in a novel manner. Combining an RGB camera, ...
Let's Resonate: How to Elicit Improvisation and Letting Go in Interactive Digital Art
MOCO '20: Proceedings of the 7th International Conference on Movement and ComputingParticipatory art allows for the spectator to be a participant or a viewer able to engage actively with interactive art. Real-time technologies offer new ways to create participative artworks. We hereby investigate how to engage participation through ...
Comments