Encumbered Interaction: a Study of Musicians Preparing to Perform

ABSTRACT

Guitars are physical instruments that require skillful two-handed use. Their use is also supported by diverse digital and physical resources, such as videos and chord charts. To understand the challenges of interacting with supporting resources at the same time as playing we conducted an ethnographic study of the preparation activities of working musicians. We observe successive stages of individual and collaborative preparation, in which working musicians engage with a diverse range of digital and physical resources to support their preparation. Interaction with this complex ecology of digital and physical resources is finely interwoven into their embodied musical practices, which are usually encumbered by having their instrument in hand, and often by playing. We identify challenges for augmenting guitars within the rehearsal process by supporting interaction that is encumbered, contextual and connected, and suggest a range of possible responses.

References

1. Ahmed Ahmed, Steve Benford, and Andy Crabtree. 2012. Digging in the crates: an ethnographic study of DJS'work. In Proceedings of the SIGCHI conference on human factors in computing systems, 1805-- 1814.
2. Luigi Atzori, Antonio Iera, and Giacomo Morabito. 2014. From "smart objects" to "social objects": The next evolutionary step of the internet of things. IEEE Communications Magazine 52, 1: 97-- 105. https://doi.org/10/gd7d7m
3. Steve Benford, Adrian Hazzard, Alan Chamberlain, Kevin Glover, Chris Greenhalgh, Liming Xu, Michaela Hoare, and Dimitrios Darzentas. 2016. Accountable artefacts: the case of the Carolan guitar. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 1163--1175.
4. Steve Benford, Adrian Hazzard, Alan Chamberlain, and Liming Xu. 2015. Augmenting a Guitar with Its Digital Footprint. In Proceedings of the international conference on New Interfaces for Musical Expression (NIME 2015). The School of Music and the Center for Computation and Technology (CCT), Louisiana State University, Baton Rouge, Louisiana, USA, 303--306.
5. Steve Benford, Peter Tolmie, Ahmed Y. Ahmed, Andy Crabtree, and Tom Rodden. 2012. Supporting traditional music-making: designing for situated discretion. In Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work, 127--136.
6. Nicolas Bouillot, Michael Wozniewski, Zack Settel, and Jeremy R. Cooperstock. 2008. A Mobile Wireless Augmented Guitar. In Proceedings of the 2008 New Interfaces for Musical Expression Conference (NIME), 189--192.
7. John Seely Brown and Paul Duguid. 2000. The Social Life of Information (Book Review). Harvard Business Review 78, 6: 194--194.
8. Anne-Marie Burns, Sébastien Bel, and Caroline Traube. 2017. Learning to play the guitar at the age of interactive and collaborative Web technologies. In Proceedings of the 14th Sound and Music Computing Conference, July 5--8, Espoo, Finland.
9. Andrew Crabtree, Mark Rouncefield, and Peter Tolmie. 2012. Doing design ethnography. Springer.
10. Anind K. Dey, Gregory D. Abowd, and Daniel Salber. 2001. A Conceptual Framework and a Toolkit for Supporting the Rapid Prototyping of Context-aware Applications. Human-Computer Interaction. 16, 2: 97--166. https://doi.org/10/b3ww6m
11. Paul Dourish. 2004. What we talk about when we talk about context. Personal and ubiquitous computing 8, 1: 19--30. https://doi.org/10/cfdfd6
12. Robert R Faulkner and Howard S Becker. 2009. "Do you know...?": The jazz repertoire in action. University of Chicago Press.
13. Rebecca Fiebrink and Perry R Cook. 2010. The Wekinator: a system for real-time, interactive machine learning in music. In Proceedings of The Eleventh International Society for Music Information Retrieval Conference (ISMIR 2010) (Utrecht).
14. Clifford Geertz. 2008. Thick description: Toward an interpretive theory of culture. In The Cultural Geography Reader. Routledge, 41-- 51.
15. Lucy Green. 2017. How popular musicians learn: A way ahead for music education. Routledge.
16. Chris Greenhalgh, Steve Benford, Adrian Hazzard, and Alan Chamberlain. 2017. Playing fast and loose with music recognition. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, 4302--4313.
17. Susumu Harada, Jacob O Wobbrock, and James A Landay. 2007. Voicedraw: a hands-free voice-driven drawing application for people with motor impairments. In Proceedings of the 9th international ACM SIGACCESS conference on Computers and accessibility, 27--34.
18. Michaela Hoare, Steve Benford, Rachel Jones, and Natasa MilicFrayling. 2014. Coming in from the margins: amateur musicians in the online age. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1295--1304.
19. Kallionpaa, Maria and Greenhalgh, Chris and Hazzard, Adrian and Weigl, David M and Page, Kevin R and Benford, Steve (2017) Composing and Realising a Game-like Performance for Disklavier and Electronics. In Proceedings of 2017 New Interfaces for Musical Expression (NIME), 15--18 May 2017, Copenhagen, Denmark.
20. Shaun K Kane, Jacob O Wobbrock, and Ian E Smith. 2008. Getting off the treadmill: evaluating walking user interfaces for mobile devices in public spaces. In Proceedings of the 10th international conference on Human computer interaction with mobile devices and services, 109--118.
21. Gerd Kortuem, Fahim Kawsar, Vasughi Sundramoorthy, and Daniel Fitton. 2010. Smart objects as building blocks for the internet of things. IEEE Internet Computing 14, 1: 44--51. https://doi.org/10/ckxwb3
22. Otso Lähdeoja. 2008. An Approach to Instrument Augmentation: the Electric Guitar. In Proceedings of the 2008 New Interfaces for Musical Expression Conference (NIME), 53--56.
23. Duncan MacConnell, Shawn Trail, George Tzanetakis, Peter Driessen, Wyatt Page, and NZ Wellington. 2013. Reconfigurable autonomous novel guitar effects (range). In Proceedings of the International Conference on Sound and Music Computing (SMC).
24. Sean McGrath, Alan Chamberlain, and Steve Benford. 2016. The Grime scene: social media, music, creation and consumption. In Proceedings of the Audio Mostly 2016, 245--250.
25. Andrew P. McPherson and Youngmoo Kim. 2010. Augmenting the Acoustic Piano with Electromagnetic String Actuation and Continuous Key Position Sensing. In Proceedings of the 2010 New Interfaces for Musical Expression Conference (NIME), 217--222.
26. Eduardo Reck Miranda and Marcelo M Wanderley. 2006. New digital musical instruments: control and interaction beyond the keyboard. Volume 21 of the Computer Music and Digital Audio Series. A-R Editions, Inc., Middleton, WI, 2006. ISBN 0--895790585-X.
27. Mark W Newman, Shahram Izadi, W Keith Edwards, Jana Z Sedivy, and Trevor F Smith. 2002. User interfaces when and where they are needed: an infrastructure for recombinant computing. In Proceedings of the 15th annual ACM symposium on User interface software and technology, 171--180.
28. Dan Newton and Mark T Marshall. 2011. Examining How Musicians Create Augmented Musical Instruments. In Proceedings of the 2011 New Interfaces for Musical Expression Conference (NIME), 155--160.
29. Alexander Ng, Stephen A Brewster, and John H Williamson. 2014. Investigating the effects of encumbrance on one-and two-handed interactions with mobile devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1981--1990.
30. Jieun Oh, Jorge Herrera, Nicholas J. Bryan, Luke Dahl, and Ge Wang. 2010. Evolving the mobile phone orchestra. In Proceedings of the International Conference on New Interfaces for Musical Expression, 82--87.
31. Antti Oulasvirta and Joanna Bergstrom-Lehtovirta. 2011. Ease of juggling: studying the effects of manual multitasking. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 3103--3112.
32. Loïc Reboursière, Christian Frisson, Otso Lähdeoja, John A. Mills, Cécile Picard-Limpens, and Todor Todoroff. 2010. Multimodal Guitar: A Toolbox For Augmented Guitar Performances. In Proceedings of the 2010 New Interfaces for Musical Expression Conference (NIME), 415--418.
33. Yvonne Rogers. 2004. New theoretical approaches for human? computer interaction. Annual review of information science and technology 38, 1: 87--143. https://doi.org/10/cdt5mk
34. Lucy A. Suchman. 1987. Plans and situated actions: The problem of human-machine communication. Cambridge university press.
35. Atau Tanaka. 2010. Mapping out instruments, affordances, and mobiles. In Proceedings of the 2010 New Interfaces for Musical Expression Conference (NIME), Sydney, Australia.
36. Luca Turchet. 2018. Smart Musical Instruments: vision, design principles, and future directions. IEEE Access: 1--1.
37. Luca Turchet, Carlo Fischione, and Mathieu Barthet. 2017. Towards the Internet of Musical Things. In Proceedings of the Sound and Music Computing Conference, 13--20.
38. Luca Turchet, Andrew McPherson, and Carlo Fischione. 2016. Smart instruments: Towards an ecosystem of interoperable devices connecting performers and audiences. In Proceedings of the Sound and Music Computing Conference, 498--505.
39. Zacharias Vamvakousis and Rafael Ramirez. 2016. The EyeHarp: A Gaze-Controlled Digital Musical Instrument. Frontiers in psychology 7: 906. https://doi.org/10/gd7hc9
40. Janice Waldron. 2013. User-generated content, YouTube and participatory culture on the Web: Music learning and teaching in two contrasting online communities. Music Education Research 15, 3: 257--274. https://doi.org/10/gdtg66
41. ULTIMATE GUITAR TABS. 1,100,000 songs catalog with free Chords, Guitar Tabs, Bass Tabs, Ukulele Chords and Guitar Pro Tabs! Retrieved September 21, 2018 from https://www.ultimateguitar.com/
42. SENSUS Smart Guitar by MIND Music Labs. MINDMusicLabs.com. Retrieved September 18, 2018 from https://www.mindmusiclabs.com/sensus/
43. Ultimate Guitar: Chords & Tabs. App Store. Retrieved September 20, 2018 from https://itunes.apple.com/gb/app/ultimate-guitar-chordstabs/id357828853?mt=8

Index Terms

• Applied computing

  • Arts and humanities

    • Sound and music computing

• Human-centered computing

  • Collaborative and social computing

    • Collaborative and social computing design and evaluation methods

      • Ethnographic studies