Steve Gibson
ABSTRACT
Opto-Phono-Kinesia (OPK) is an audio-visual performance piece in which all media elements are controlled by the body movements of a single performer. The title is a play on a possible synesthetic state involving connections between vision, sound and body motion. Theoretically, for a person who experiences this state, a specific colour could trigger both a sound and a body action. This synesthetic intersection is simulated in OPK by simultaneity of body movement, and audio-visual result.
Using the Gesture and Media System 3.0 motion-tracking system, the performer can dynamically manipulate an immersive environment using two small infrared trackers. The project employs a multipart interface design based on a formal model of increasing complexity in visual-sound-body mapping, and is therefore best performed by an expert performer with strong spatial memory and advanced musical ability. OPK utilizes the "body as experience, instrument and interface" [1] for control of a large-scale environment.
- ACM. TEI Arts Track website. 2018. Retrieved October 10, 2017 from https://tei.acm.org/2018/arts-track/Google Scholar
- Cretien van Campen. 2007. The Hidden Sense: Synesthesia in Art and Science. Massachusetts Institute of Technology (MIT), Cambridge, MA, 1.Google Scholar
- Cretien van Campen. 2007. The Hidden Sense: Synesthesia in Art and Science. Massachusetts Institute of Technology (MIT), Cambridge, MA, 49.Google Scholar
- Steve Gibson, Virtual DJ website. 2006. Retrieved October 11, 2017 from http://www.telebody.ws/VirtualDJGoogle Scholar
Index Terms
- Opto-Phono-Kinesia (OPK): Designing Motion-Based Interaction for Expert Performers
Recommendations
SoLu: hyperinstrument
TEI '11: Proceedings of the fifth international conference on Tangible, embedded, and embodied interactionDemonstration of a proposal of correspondence between Light and Sound. This proposal is materially exemplified by means of a new hyperinstrument, which gives its users the control over a multi-sensorial algorithmic composition generated in real-time. ...
A novel method to musicalize shape and visualize music and a novel technique in music cryptography
AbstractSince many years ago, musicians have composed music based on the images that they have had in their minds. On the other hand, music affects people’s imagination while hearing it. This research provides a method that can transform shape to music ...
DJuggling: Sonification of expressive movement performance
SIGGRAPH '23: ACM SIGGRAPH 2023 Real-Time Live!In the real-time demo, we demonstrate how to create a musical performance using juggling movement as an instrument. We have equipped juggling balls with accelerometers, gyroscopes, and WiFi sensors. The system measures acceleration and rotation in a ...
Comments