Content-based music structure analysis with applications to music semantics understanding

ABSTRACT

In this paper, we present a novel approach for music structure analysis. A new segmentation method, beat space segmentation, is proposed and used for music chord detection and vocal/instrumental boundary detection. The wrongly detected chords in the chord pattern sequence and the misclassified vocal/instrumental frames are corrected using heuristics derived from the domain knowledge of music composition. Melody-based similarity regions are detected by matching sub-chord patterns using dynamic programming. The vocal content of the melody-based similarity regions is further analyzed to detect the content-based similarity regions. Based on melody-based and content-based similarity regions, the music structure is identified. Experimental results are encouraging and indicate that the performance of the proposed approach is superior to that of the existing methods. We believe that music structure analysis can greatly help music semantics understanding which can aid music transcription, summarization, retrieval and streaming.

References

1. Bartsch, M. A., and Wakefield, G. H. To Catch a Chorus: Using Chroma-based Representations for Audio Thumbnailing. In Proc. WASPA. 2001.
2. Berenzweig, A. L., and Ellis, D.P.W. Location singing voice segments within music signals. In Proc. IEEE WASPAA. 2001.
3. Chai, W., and Vercoe, B. Music Thumbnailing via Structural Analysis. In Proc. ACM Multimedia. 2003, 223--226.
4. Cooper, M., and Foote, J. Automatic Music Summarization via Similarity Analysis. In Proc. ISMIR. 2002.
5. Collobert <http://www.idiap.ch/~collober>, R., and Bengio <http://www.idiap.ch/~bengio>, S. SVMTorch: Support Vector Machines for Large-Scale Regression Problems <http://www.ai.mit.edu/projects/jmlr/papers/volume1/collobert01a/collobert01a.ps.gz>. Journal of Machine Learning Research. 2001, Vol 1, 143--160.
6. Duxburg. C, Sandler. M., and Davies. M. A Hybrid Approach to Musical Note Onset Detection. In Proc. International Conference on DAFx. 2002.
7. Foote, J., Cooper, M., and Girgensohn, A. Creating Music Video using Automatic Media Analysis. In Proc. ACM Multimedia. 2002.
8. Fujinaga, I. Machine Recognition of Timbre Using Steady-state Tone of Acoustic Musical Instruments. In Proc. ICMC. 1998, 207--210.
9. Ghias, A., Logan, J., Chamberlin, D., and Smith, B. C. Query By Humming: Musical Information Retrieval in an Audio Database. In Proc. ACM Multimedia. 1995, 231--236.
10. Goto, M. A Chorus-Section Detecting Method for Musical Audio Signals. In Proc. IEEE ICASSP. 2003.
11. Goto, M. An Audio-based Real-time Beat Tracking System for Music With or Without Drum-sounds. Journal of new Music Research. June. 2001, Vol.30, 159--171.
12. Deller, J. R., Hansen, J.H.L., and Proakis, H. J. G. Discrete-Time Processing of Speech Signals, IEEE Press, 2000.
13. Kim, Y.K., and Brian, Y. Singer Identification in Popular Music Recordings Using Voice Coding Features. In Proc. ISMIR 2002.
14. Logan, B., and Chu, S. Music Summarization Using Key Phrases. In Proc. IEEE ICASSP. 2000.
15. Lu, L., and Zhang, H. Automated Extraction of Music Snippets. In Proc. ACM Multimedia. 2003, 140--147.
16. Navarro, G. A guided tour to approximate string matching, ACM Computing Surveys, March 2001, Vol.33, No 1, 31--88.
17. Rossing, T.D., Moore, F. R., and Wheeler, P. A. Science of Sound. Addison Wesley, 3rd edition 2001.
18. Rudiments and Theory of Music. The associated board of the royal schools of music, 14 Bedford Square, London, WC1B 3JG, 1949.
19. Scheirer, E. D. Tempo and Beat Analysis of Acoustic Musical Signals. Journal of the Acoustical Society of America. January 1998, Vol 103, No 1, 588 -- 601.
20. Sheh, A., and Ellis, D.P.W. Chord Segmentation and Recognition using EM-Trained Hidden Markov Models. In Proc. ISMIR 2003.
21. Shenoy, A., Mohapatra, R., and Wang, Y. Key Detection of Acoustic Musical Signals, In Proc, ICME 2004.
22. Ten Minute Master No 18: Song Structure. MUSIC TECH magazine. www.musictechmag.co.uk <http://www.musictechmag.co.uk> (Oct. 2003), 62 -- 63.
23. Wang, Y. et al. Content-Based UEP: A New Scheme for Packet Loss Recovery in Music Streaming. In Proc. ACM Multimedia. 2003. 412 -- 421.
24. Xu, C.S., Maddage, N.C., and Shao, X. Automatic Music Classification and Summarization. In IEEE Transaction on Speech and Audio Processing (accepted).
25. Young, S. et al. The HTK Book. Dept of Engineering, University of Cambridge, Version 3.2, 2002.

Index Terms

• Information systems

  • Information retrieval

    • Document representation

    • Search engine architectures and scalability

      • Search engine indexing

• Theory of computation

  • Semantics and reasoning

    • Program reasoning

      • Abstraction