Gil Weinberg
ABSTRACT
We discuss our ongoing research in sonification of neural activity as demonstrated in the "Fish and Chips" and "BrainWaves" projects. We argue that sonification can serve as an effective technique for the representation of complex spatial information such as neural activity due to the auditory system's ability to perceive stimuli at a wide spatial cover and its inclination to perceive spatial patterns in sonic input. The paper discusses aesthetic and functional aspects of sonification in this context and describes the evolution of our technique, artistic approach, and interaction design -- from the low-resolution graphical user interface in "Fish and Chips" to the high-resolution tangible interaction with newly developed controllers in "BrainWaves". We conclude with an evaluative discussion and a number of suggestions for future work.
- Ben-Tal, O. and Berger, J. Creative aspects of sonification, Leonardo, 20 (3), MIT Press 2004, 229--232.Google Scholar
- Catts, O., Zurr, I. Growing Semi-Living Sculptures Leonardo 35(4), MIT Press 2002Google Scholar
- Derthick, M., Kolejejchick, J., Roth, S. F. An interactive visualization environment for data exploration. Proc. Of Knowledge Discovery in Databases. AAAI Press, August 1997, 2--9.Google Scholar
- Dibben, N. The Perception of Structural Stability in Atonal Music: The Influence of Salience, Stability, Horizontal Motion, Pitch Commonality, and Dissonance. Music Perception, 16(3), 1999, 265--294.Google ScholarCross Ref
- Fry, B. (2002). Genome Valence. access at: http://acg.media.mit.edu/people/fry/valence/.Google Scholar
- Heer, J., Card, S. K., Landay, J. A. Prefuse: A toolkit for interactive information visualization. Proc. of the International Conference for Computer-Human Interaction (CHI 2005), ACM Press (2005), 421--430. Google ScholarDigital Library
- Hunt, A. and Hermann, T. The importance of interaction in sonification. Proc. of the International Conference on Auditory Display (ICAD 2004) 61--64.Google Scholar
- Lodha, S. K., Beahan, J., Heppe, T., Joseph, A. J., and Zane-Ulman, B. Muse: A musical data sonification toolkit. Proc. of the International Conference on Auditory Display, ICAD 1997, 41--54.Google Scholar
- Paine, G. (2004). Reeds -- a responsive sound installation. Proc. of the International Conference on Auditory Display (ICAD 2004)Google Scholar
- Paine, G., Ramakrishnan, C., Tydeman, J. (2005). Meterosonics. Access at: http://www.meterosonics.comGoogle Scholar
- Potter's Neuroengineering Laboratory at Georgia Tech - http://www.neuro.gatech.edu/groups/potter/index.html.Google Scholar
- Schmuckler, M. A. Testing Models of Melodic Contour Similarity. Music Perception, 16(3), 1999, 295--326.Google ScholarCross Ref
- Sturm, B. L. Surf Music: Sonification of Ocean Buoy SpectralData. Proc. of the 2002 International Conference on Auditory Display, (ICAD 2002) Kyoto, Japan, 2002.Google Scholar
- Symbiotica Research Group - http://www.symbiotica.uwa.edu.au/Google Scholar
- Wagenaar, D. A., Madhavan, R., Pine, J., and Potter, S. M. Controlling bursting in cortical cultures with closed-loop multi-electrode stimulation. Journal of Neuroscience, 25(3), January 19, 2005, 680--688.Google ScholarCross Ref
- Walker, B. and Cothran, J. Sonification sandbox: a graphical toolkit for auditory graphs. Proc. of the International Conference on Auditory Display (ICAD 2003).Google Scholar
- Yeo, W. S., Berger, J., and Zune Lee. SonART: A framework for data sonification, visualization and networked multimedia applications. Proc. of the International Computer Music Conference (ICMC 2004).Google Scholar
Index Terms
- Interactive sonification of neural activity
Recommendations
Aesthetic strategies in sonification
Sound can be listened to in various ways and with different intentions. Multiple factors influence how and what we perceive when listening to sound. Sonification, the acoustic representation of data, is in essence just sound. It functions as ...
Evaluation of psychoacoustic sound parameters for sonification
ICMI '17: Proceedings of the 19th ACM International Conference on Multimodal InteractionSonification designers have little theory or experimental evidence to guide the design of data-to-sound mappings. Many mappings use acoustic representations of data values which do not correspond with the listener's perception of how that data value ...
The sonification space
Sonification is a fairly new term to scientists who are unaware of its multiple use cases. Even if some general definitions of the concept of sonification are commonly accepted, heterogeneous techniques - significantly different as it regards approaches,...
Comments