Effect of accompanying onomatopoeia to interaction sound for altering user perception in virtual reality

ABSTRACT

Onomatopoeia refers to a word that phonetically imitates, resembles the sound, or depict an event at hand. In languages like Korean and Japanese, it is used in everyday conversation to emphasize certain situation and enrich the prose. In this poster, we explore if the use of onomatopoeia, visualized and added to the usual sound feedback, could be taken advantage to increase or alter the perceived realism of the sound feedback itself, and furthermore of the situation at hand in virtual reality. A pilot experiment was run to compare the user's subjective perceived realism and experience under four test conditions of presenting a simple physical interaction, accompanying it with: (1) just the "as-is" sound (baseline), (2) "as-is" sound and onomatopoeia, (3) a representative sound sample (e.g. one for all different collision conditions), and (4) a representative sound sample and onomatopoeia. Our pilot study has found that the use of onomatopoeia can alter and add on to the perceived realism/naturalness of the virtual situation such that the experiences of the single representative sound added with the onomatopoeia and "as-is" sound were deemed similar.

References

1. Wikipedia, Onomatopoeia, Retrieved 2018, https://en.wikipedia.org/wiki/Onomatopoeia
2. Tsukasa Fukusato and Shigeo Morishima. 2014. Automatic depiction of onomatopoeia in animation considering physical phenomena. In Proceedings of the Seventh International Conference on Motion in Games. ACM, 161--169.
3. Fangzhou Wang, Hidehisa Nagano, Kunio Kashino, and Takeo Igarashi. 2017. Visualizing video sounds with sound word animation to enrich user experience. IEEE Transactions on Multimedia 19, 2 (2017), 418--429.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction paradigms

      • Virtual reality