The Story Behind the&nbsp;RWC Music Database:&nbsp;An Interview with Masataka&nbsp;Goto

Meinard Müller; Stefan Balke; Masataka Goto

doi:10.5334/tismir.261

Abstract

The Real World Computing (RWC) Music Database has been a widely used and valuable resource in music information retrieval (MIR) research for over two decades, offering high‑quality audio recordings and comprehensive annotations for a variety of MIR tasks. In light of recent efforts to transform the RWC into an open‑domain and community‑driven dataset, this editorial presents a revised transcript of an interview with Masataka Goto, one of the original creators of the RWC. Conducted on February 18, 2025, by Stefan Balke and Meinard Müller, the interview explores the origins of the RWC, the motivations behind its development, the challenges involved in its production and annotation, and its long‑term impact on MIR research. The conversation also reflects on the evolution of dataset practices in MIR and discusses future directions for open, collaborative dataset initiatives. This transcript has been carefully edited for clarity while preserving the authenticity of the questions and responses. Beyond its immediate relevance to the RWC, the interview provides a unique historical perspective on MIR, capturing firsthand insights from a key figure whose work has significantly shaped the field. By making this interview available, we aim to contribute to broader conversations on dataset sustainability, open access, and the role of community‑driven research in MIR.

References

Bittner, R. M., Salamon, J., Tierney, M., Mauch, M., Cannam, C., and Bello, J. P. (2014). MedleyDB: A multitrack dataset for annotation‑intensive MIR research. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR), Taipei, Taiwan (pp. 155–160).
Back to article
Böck, S., Krebs, F., and Widmer, G. (2016). Joint beat and downbeat tracking with recurrent neural networks. In Proceedings of the International Societyfor Music Information Retrieval Conference (ISMIR), New York City, New York, USA (pp. 255–261).
Back to article
Boulanger, R., Mathews, M., Vercoe, B., and Dannenberg, R. (1990). Conducting the MIDI orchestra, Part 1: Interviews with Max Mathews, Barry Vercoe, and Roger Dannenberg. Computer Music Journal, 14(2), 34–46.
Back to article
Cannam, C., Landone, C., and Sandler, M. B. (2010). Sonic Visualiser: An open source application for viewing, analysing, and annotating music audio files. In Proceedings of the International Conference on Multimedia, Florence, Italy (pp. 1467–1468).
Back to article
Cheng, T., and Goto, M. (2023). Transformer‑based beat tracking with a low‑resolution encoder and a high‑resolution decoder. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR), Milan, Italy (pp. 466–473).
Back to article
Cho, T., and Bello, J. P. (2011). A feature smoothing method for chord recognition using recurrence plots. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR), Miami, Florida, USA (pp. 651–656).
Back to article
Davies, M. E. P., Hamel, P., Yoshii, K., and Goto, M. (2014). AutoMashUpper: Automatic creation of multi‑song music mashups. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 22(12), 1726–1737.
Back to article
Dessein, A., Cont, A., and Lemaitre, G. (2010). Real‑time polyphonic music transcription with non‑negative matrix factorization and beta‑divergence. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR), Utrecht, The Netherlands (pp. 489–494).
Back to article
Ewert, S., Müller, M., and Grosche, P. (2009). High resolution audio synchronization using chroma onset features. In Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Taipei, Taiwan (pp. 1869–1872).
Back to article
Goto, M. (2000). A robust predominant‑F0 estimation method for real‑time detection of melody and bass lines in CD recordings. In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) (Vol. 2, pp. 757–760).
Back to article
Goto, M. (2003a). A chorus‑section detecting method for musical audio signals. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Hong Kong, China (pp. 437–440).
Back to article
Goto, M. (2003b). Music scene description project: Toward audio‑based real‑time music understanding. In Proceedings of the International Conference on Music Information Retrieval (ISMIR) (pp. 231–232).
Back to article
Goto, M. (2003c). SmartMusicKIOSK: Music listening station with chorus‑search function. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST) (pp. 31–40).
Back to article
Goto, M. (2004). Development of the RWC music database. In Proceedings of the International Congress on Acoustics (ICA) (pp. 553–556).
Back to article
Goto, M. (2006a). AIST Annotation for the RWC Music Database. In Proceedings of the International Conference on Music Information Retrieval (ISMIR) (pp. 359–360).
Back to article
Goto, M. (2006b). A chorus section detection method for musical audio signals and its application to a music listening station. IEEE Transactions on Audio, Speech, and Language Processing, 14(5), 1783–1794.
Back to article
Goto, M., and Dannenberg, R. B. (2019). Music interfaces based on automatic music signal analysis: New ways to create and listen to music. IEEE Signal Processing Magazine, 36(1), 74–81.
Back to article
Goto, M., Hashiguchi, H., Nishimura, T., and Oka, R. (2002). RWC Music Database: Popular, classical and jazz music databases. In Proceedings of the International Conference on Music Information Retrieval (ISMIR), Paris, France (pp. 287–288).
Back to article
Goto, M., Hashiguchi, H., Nishimura, T., and Oka, R. (2003). RWC Music Database: Music genre database and musical instrument sound database. In Proceedings of the International Conference on Music Information Retrieval (ISMIR), Baltimore, Maryland, USA (pp. 229–230).
Back to article
Goto, M., and Muraoka, Y. (1997). Issues in evaluating beat tracking systems. In Working Notes of the IJCAI‑97 Workshop on Issues in AI and Music‑Evaluation and Assessment (pp. 9–16).
Back to article
Goto, M., Yoshii, K., Fujihara, H., Mauch, M., and Nakano, T. (2011). Songle: A web service for active music listening improved by user contributions. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR), Miami, Florida, USA (pp. 311–316).
Back to article
Grosche, P., and Müller, M. (2011). Extracting predominant local pulse information from music recordings. IEEE Transactions on Audio, Speech, and Language Processing, 19(6), 1688–1701.
Back to article
Harte, C., and Sandler, M. B. (2005). Automatic chord identification using a quantised chromagram. In Proceedings of the Audio Engineering Society Convention, Barcelona, Spain.
Back to article
Itoyama, K., Kitahara, T., Komatani, K., Ogata, T., and Okuno, H. G. (2006). Automatic feature weighting in automatic transcription of specified part in polyphonic music. In Proceedings of the International Conference on Music Information Retrieval (ISMIR), Victoria, Canada (pp. 172–175).
Back to article
Joder, C., Essid, S., and Richard, G. (2011). A conditional random field framework for robust and scalable audio‑to‑score matching. IEEE Transactions on Audio, Speech, and Language Processing, 19(8), 2385–2397.
Back to article
Korzeniowski, F., and Widmer, G. (2016). Feature learning for chord recognition: The deep chroma extractor. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR), New York City, New York, USA (pp. 37–43).
Back to article
Mauch, M., and Dixon, S. (2014). pYIN: A fundamental frequency estimator using probabilistic threshold distributions. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Florence, Italy (pp. 659–663).
Back to article
McFee, B., Kim, J. W., Cartwright, M., Salamon, J., Bittner, R. M., and Bello, J. P. (2019). Open‑source practices for music signal processing research: Recommendations for transparent, sustainable, and reproducible audio research. IEEE Signal Processing Magazine, 36(1), 128–137.
Back to article
Nakano, T., and Goto, M. (2009). VocaListener: A singing‑to‑singing synthesis system based on iterative parameter estimation. In Proceedings of the 6th Sound and Music Computing Conference (SMC) (pp. 343–348).
Back to article
Otsu, N. (1993). Real world computing program ‑ overview. In Proceedings of the International Joint Conference on Neural Networks (IJCNN), Nagoya, Japan (pp. 1065–1068).
Back to article
Paulus, J., and Klapuri, A. P. (2009). Music structure analysis using a probabilistic fitness measure and a greedy search algorithm. IEEE Transactions on Audio, Speech, and Language Processing, 17(6), 1159–1170.
Back to article
Roads, C., and Minsky, M. (1980). Interview with Marvin Minsky. Computer Music Journal, 4(3), 25–39.
Back to article
Ryynänen, M., and Klapuri, A. (2005). Polyphonic music transcription using note event modeling. In Proceedings of the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), New Paltz, New York, USA (pp. 319–322).
Back to article
Serra, X. (2014). Creating research corpora for the computational study of music: The case of the CompMusic project. In Proceedings of the AES International Conference on Semantic Audio, London, UK.
Back to article
Wang, J.‑C., Hung, Y.‑N., and Smith, J. B. L. (2022). To catch a chorus, verse, intro, or anything else: Analyzing a song with structural functions. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) (pp. 416–420).
Back to article
Wu, Y., and Li, W. (2019). Automatic audio chord recognition with MIDI‑trained deep feature and BLSTM‑CRF sequence decoding model. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 27(2), 355–366.
Back to article

The Story Behind the RWC Music Database: An Interview with Masataka Goto

Abstract

Paradigm

My account