A playback system that synchronizes the musical phrases with listener's respiration phases

ABSTRACT

We propose a novel sound presentation system that utilizes a listener's respiration (breath) information. The system uses sound data, targets of the respiration phase bounded to the sound data and the listener's respiration trace. In replaying the sound, the system attempts to change the replay speed to minimize the difference between the target of respiration phase and the observed (listener's) respiration phase. Thus, a listener using the system has more chances to listen to a specific phrase with a specific respiration phase. In an experiment, chromatic scale movements were presented with different replay strategies to evaluate the system. Although the participant did not know about the control system, they reported a difference in their arousal feeling. The results indicate that music presentation that follows the listener's respiration will have significant effect on the impression of sound, suggesting that we may extend the possibility of sound presentation.

References

1. Bernardi, L., Porta, C., Casucci, G., Balsamo, R., Bernardi, N. F., Fogari, R., and Sleight, P. Dynamic interactions between musical, cardiovascular, and cerebral rhythms in humans. Circulation 119, (2009), 3171--3180.
2. George, R., Cung, T. D., Vedam, S. S., Ramakrishnan, V., Mohan, R., Paul J. K. E. W. Audio-visual biofeedback for respiratory-gated radio therapy: impact of audio instruction and audio-visual biofeedback on respiratory-gated radio therapy. Int. J. Radiation Oncology Biol. Phys 65, (2006), 924--933.
3. Haas, F, Distenfeld, S. and Axen, K. Effects of perceived musical rhythm on respiratory pattern. J Appl Physiol 61, (1986), 11851191.
4. Kamamoto, Y., Moriya, T., and Harada, N. Multichannel Linear Prediction Method Compliant with the MPEG-4 ALS. IEICE. Trans. Fundamentals, E91-A (2008), 756--762.
5. Lin, Y. P., Wang, C. H., Wu, T. L., Jeng, S.-K., and Chen, J.-H. Support vector machine for EEG signal classification during listening to emotional music. Proc. IEEE 10th Workshop on Multimedia Signal Processing, (2008), 127--130.
6. Sato, T. G., Ohsuga, and M., Moriya, T. Increase in the timing coincidence of a respiration event induced by listening repeatedly to the same music track. Acoustical Science and Technology 33, 4 (2012), 255--261.
7. Sato, T. G., Ooishi, Y. Sound presentation during different respiration phases alters the sound-induced vasoconstriction mediated by the sympathetic nerve, Neuroscience Letters 521, (2012), 67--70.
8. Schafer, C., Rosenblum, M. G. Kurths, J., and Abel, H. Heartbeat synchronized with ventilation. Science, (1998), 239--240.
9. Yamamoto, T., Miyake, Y. Analysis of interaction in musical communication and its modeling. Proc. IEEE International Conference on Systems Man and Cybernetics, (2000), 763--768.

Index Terms

• Applied computing

  • Arts and humanities

    • Sound and music computing

• Human-centered computing

  • Human computer interaction (HCI)

• Information systems

  • Information retrieval

    • Specialized information retrieval

      • Multimedia and multimodal retrieval

        • Music retrieval