Spatial audio: graphic modeling for X3D

ABSTRACT

In audio-graphic scenes, visual and audio modalities are synchronized in time and space, and their behaviour is determined by a common process. We present here a novel way of modeling audio-graphic content for interactive 3D scenes with the concept of sound processes and their activation through 2D or 3D profiles. Many 3D applications today support both graphical and audio effects to provide a more realistic user experience; however a common model and interchange format for interactive audio-graphic scenes is still lacking. X3D is one of the most promising formats for 3D scene representation. It is extensible and supports simple spatial audio representation and almost all basic and advanced 3D computer graphics techniques. We therefore propose an extension of the X3D standard to represent the sound process and activation profile model for providing a rich audio-graphic description in X3D.

References

1. Bonneel, N., Drettakis, G., Tsingos, N., Viaud-Delmon, I., and James, D. 2008. Fast modal sounds with scalable frequency-domain synthesis. ACM Transactions on Graphics (TOG) 27, 3, 24.
2. Freed, A., MacCallum, J., Schmeder, A., and Wessel, D. 2010. Visualizations and Interaction Strategies for Hybridization Interfaces. In Proceedings of the International Conference for New Instruments for Musical Expression NIME, 343--347.
3. Funkhouser, T. A., Min, P., and Carlbom, I. 1999. Realtime acoustic modeling for distributed virtual environments. In Proceedings of SIGGRAPH 99, 365--374.
4. Moeck, T., Bonneel, N., Tsingos, N., Drettakis, G., Viaud-Delmon, I., and Aloza, D. 2007. Progressive perceptual audio rendering of complex scenes. In ACM SIGGRAPH Symp. on Interactive 3D Graphics and Games.
5. O'Brien, J. F., Shen, C., and Gatchalian, C. M. 2002. Synthesizing sounds from rigid-body simulations. In ACM SIGGRAPH/Eurographics Symp. on Computer animation, 175--181.
6. Schwarz, D., Cahen, R., Ding, H., and Jacquemin, C. 2011. Sound level of detail in interactive audiographic 3D scenes. In Procees of the International Computer Music Conference (ICMC).
7. The Web3D Consortium, 2008. X3D specification -- part 1: Architecture and base components. http://www.web3d.org/x3d/specifications. Edition 2, ISO/IEC 19775--1.2:2008.
8. Tsingos, N., Funkhouser, T., Ngan, A., and Carlbom, I. 2001. Modeling acoustics in virtual environments using the uniform theory of diffraction. In Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH, 545--552.
9. Tsingos, N., Gallo, E., and Drettakis, G. 2004. Perceptual audio rendering of complex virtual environments. In ACM SIGGRAPH 2004 Papers, ACM, 249--258.

Index Terms

• Applied computing

  • Arts and humanities

    • Sound and music computing

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Image and video acquisition

        • 3D imaging

  • Computer graphics

    • Animation

• Information systems

  • Information retrieval

    • Specialized information retrieval

      • Multimedia and multimodal retrieval

        • Music retrieval

  • Information storage systems