Music Tempo Estimation: Are We Done Yet?

Hendrik Schreiber; Julián Urbano; Meinard Müller

doi:10.5334/tismir.43

Abstract

With the advent of deep learning, global tempo estimation accuracy has reached a new peak, which presents a great opportunity to evaluate our evaluation practices. In this article, we discuss presumed and actual applications, the pros and cons of commonly used metrics, and the suitability of popular datasets. To guide future research, we present results of a survey among domain experts that investigates today’s applications, their requirements, and the usefulness of currently employed metrics. To aid future evaluations, we present a public repository containing evaluation code as well as estimates by many different systems and different ground truths for popular datasets.

References

1Allen, P. E., & Dannenberg, R. B. (1990). Tracking musical beats in real time. In Proceedings of the International Computer Music Conference, pages 140–143. International Computer Music Association.
Back to article
2Alonso, M., David, B., & Richard, G. (2003). A study of tempo tracking algorithms from polyphonic music signals. In 4th COST 276 Workshop.
Back to article
3Bock, S., Davies, M. E., & Knees, P. (2019). Multitask learning of tempo and beat: Learning one to improve the other. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR), pages 486–493. Delft, The Netherlands.
Back to article
4Bock, S., Krebs, F., & Widmer, G. (2015). Accurate tempo estimation based on recurrent neural networks and resonating comb filters. In Proceedings of the 16th International Society for Music Information Retrieval Conference (ISMIR), pages 625–631. Malaga, Spain.
Back to article
5Bodoff, D. (2008). Test theory for evaluating reliability of IR test collections. Information Processing & Management, 44(3), 1117–1145. DOI: 10.1016/j.ipm.2007.11.006
Back to article
6Bosch, J. J., Marxer, R., & Gomez, E. (2016). Evaluation and combination of pitch estimation methods for melody extraction in symphonic classical music. Journal of New Music Research, 45(2), 101–117. DOI: 10.1080/09298215.2016.1182191
Back to article
7Brennan, R. L. (2003). Generalizability theory. Journal of Educational Measurement, 40(1), 105–107. DOI: 10.1111/j.1745-3984.2003.tb01098.x
Back to article
8Carterette, B., Pavlu, V., Kanoulas, E., Aslam, J. A., & Allan, J. (2009). If I had a million queries. In European Conference on Information Retrieval, pages 288–300. DOI: 10.1007/978-3-642-00958-7_27
Back to article
9Cornelis, O., Six, J., Holzapfel, A., & Leman, M. (2013). Evaluation and recommendation of pulse and tempo annotation in ethnic music. Journal of New Music Research, 42(2), 131–149. DOI: 10.1080/09298215.2013.812123
Back to article
10Cronbach, L. J., Rajaratnam, N., & Gleser, G. C. (1963). Theory of generalizability: A liberalization of reliability theory. British Journal of Statistical Psychology, 16(2), 137–163. DOI: 10.1111/j.2044-8317.1963.tb00206.x
Back to article
11Davies, M. E., Plumbley, M. D., & Eck, D. (2009). Towards a musical beat emphasis function. In Proceedings of the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WAS-PAA), pages 61–64. New Paltz, NY, USA. IEEE. DOI: 10.1109/ASPAA.2009.5346462
Back to article
12Dixon, S. (2001). Automatic extraction of tempo and beat from expressive performances. Journal of New Music Research, 30(1), 39–58. DOI: 10.1076/jnmr.30.1.39.7119
Back to article
13Downie, J. S. (2008). The music information retrieval evaluation exchange (2005–2007): A window into music information retrieval research. Acoustical Science and Technology, 29(4), 247–255. DOI: 10.1250/ast.29.247
Back to article
14Ellis, D. P., & Poliner, G. E. (2007). Identifying ‘cover songs’ with chroma features and dynamic pro-gramming beat tracking. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), volume 4, Honolulu, Hawaii, USA. DOI: 10.1109/ICASSP.2007.367348
Back to article
15Elowsson, A. (2016). Beat tracking with a cepstroid invariant neural network. In Proceedings of the 17th International Society for Music Information Retrieval Conference (ISMIR), pages 351–357. New York, NY, USA.
Back to article
16Elowsson, A., & Friberg, A. (2015). Modeling the perception of tempo. Journal of the Acoustical Society of America, 137(6), 3163–3177. DOI: 10.1121/1.4919306
Back to article
17Font, F., & Serra, X. (2016). Tempo estimation for music loops and a simple confidence measure. In Proceedings of the 17th International Society for Music Information Retrieval Conference (ISMIR), pages 269–275. New York, NY, USA.
Back to article
18Foroughmand, H., & Peeters, G. (2019). Deep-rhythm for tempo estimation and rhythm pattern recognition. In Proceedings of the International Society for Music Information Retrieval Conference (IS-MIR), pages 636–643. Delft, The Netherlands.
Back to article
19Friberg, A., & Sundberg, J. (1995). Time discrimination in a monotonic, isochronous sequence. Journal of the Acoustical Society of America, 98(5), 2524–2531. DOI: 10.1121/1.413218
Back to article
20Gartner, D. (2013). Tempo detection of urban music using tatum grid non negative matrix factorization. In Proceedings of the 14th International Society for Music Information Retrieval Conference (ISMIR), pages 311–316. Curitiba, Brazil.
Back to article
21Gkiokas, A., Katsouros, V., Carayannis, G., & Stafylakis, T. (2012). Music tempo estimation and beat tracking by applying source separation and metrical relations. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Kyoto, Japan. DOI: 10.1109/ICASSP.2012.6287906
Back to article
22Goto, M., Hashiguchi, H., Nishimura, T., & Oka, R. (2002). RWC music database: Popular, classical and jazz music databases. In Proceedings of the International Conference on Music Information Retrieval (ISMIR). Paris, France.
Back to article
23Goto, M., Hashiguchi, H., Nishimura, T., & Oka, R. (2003). RWC music database: Music genre database and musical instrument sound database. In Proceedings of the International Conference on Music Information Retrieval (ISMIR), pages 229–230. Baltimore, MD, USA.
Back to article
24Goto, M., & Muraoka, Y. (1994). A beat tracking system for acoustic signals of music. In Proceedings of the Second ACM International Conference on Multimedia, pages 365–372. San Francisco, CA, USA. DOI: 10.1145/192593.192700
Back to article
25Gouyon, F., Klapuri, A. P., Dixon, S., Alonso, M., Tzane- takis, G., Uhle, C., & Cano, P. (2006). An experimental comparison of audio tempo induction algorithms. IEEE Transactions on Audio, Speech, and Language Processing, 14(5), 1832–1844. DOI: 10.1109/TSA.2005.858509
Back to article
26Hainsworth, S. W. (2004). Techniques for the Automated Analysis of Musical Audio. PhD thesis, University of Cambridge, UK.
Back to article
27Holzapfel, A., Davies, M. E., Zapata, J. R., Oliveira, J. L., & Gouyon, F. (2012). Selective sampling for beat tracking evaluation. IEEE Transactions on Audio, Speech, and Language Processing, 20(9), 2539–2548. DOI: 10.1109/TASL.2012.2205244
Back to article
28Humphrey, E. J., Salamon, J., Nieto, O., Forsyth, J., Bittner, R. M., & Bello, J. P. (2014). JAMS: A JSON annotated music specification for reproducible MIR research. In Proceedings of the 15th International Society for Music Information Retrieval Conference (ISMIR), pages 591–596. Taipei, Taiwan.
Back to article
29Klapuri, A. P., Eronen, A. J., & Astola, J. (2006). Analysis of the meter of acoustic musical signals. IEEE Transactions on Audio, Speech and Language Processing, 14(1), 342–355. DOI: 10.1109/TSA.2005.854090
Back to article
30Knees, P., Faraldo, A., Herrera, P., Vogl, R., Bock, S., Horschlager, F., & Le Goff, M. (2015). Two data sets for tempo estimation and key detection in electronic dance music annotated from user corrections. In Proceedings of the 16th International Society for Music Information Retrieval Conference (ISMIR), pages 364–370. Malaga, Spain.
Back to article
31Knees, P., & Schedl, M. (2016). Music Similarity and Retrieval: An Introduction to Audio- and Web-based Strategies. The Information Retrieval Series. Springer Berlin Heidelberg. DOI: 10.1007/978-3-662-49722-7_1
Back to article
32Krebs, F., Bock, S., & Widmer, G. (2013). Rhythmic pattern modeling for beat and downbeat tracking in musical audio. In Proceedings of the 14th International Society for Music Information Retrieval Conference (ISMIR), pages 227–232. Curitiba, Brazil.
Back to article
33Lamere, P. (2009). In search of the click track. Blog post https://musicmachinery.com/2009/03/02/in-search-of-the-click-track/, last accessed 12/9/2018.
Back to article
34Marchand, U., & Peeters, G. (2015). Swing Ratio Estimation. In Proceedings of the International Conference on Digital Audio Effects (DAFx). Trondheim, Norway.
Back to article
35Marchand, U., & Peeters, G. (2016). The extended ballroom dataset. In Late Breaking Demo Session of the 17th International Society for Music Information Retrieval Conference (ISMIR). New York, NY, USA.
Back to article
36McKinney, M. F., Moelants, D., Davies, M. E., & Klapuri, A. P. (2007). Evaluation of audio beat tracking and music tempo extraction algorithms. Journal of New Music Research, 36(1), 1–16. DOI: 10.1080/09298210701653252
Back to article
37Moelants, D., & McKinney, M. F. (2004). Tempo perception and musical content: What makes a piece fast, slow or temporally ambiguous. In Proceedings of the 8th International Conference on Music Perception and Cognition, pages 558–562.
Back to article
38Oliveira, J. L., Gouyon, F., Martins, L. G., & Reis, L. P. (2010). IBT: A real-time tempo and beat tracking system. In Proceedings of the 11th International Society for Music Information Retrieval Conference (ISMIR), pages 291–296.
Back to article
39Peeters, G. (2005). Time variable tempo detection and beat marking. In Proceedings of the International Computer Music Conference (ICMC). Barcelona, Spain.
Back to article
40Peeters, G. (2007). Template-based estimation of time- varying tempo. EURASIP Journal on Advances in Signal Processing, 2007(1), 158–171. DOI: 10.1155/2007/67215
Back to article
41Peeters, G., & Flocon-Cholet, J. (2012). Perceptual tempo estimation using GMM-regression. In Proceedings of the Second International ACM Workshop on Music Information Retrieval with User-Centered and Multimodal Strategies (MIRUM), pages 45–50. New York, NY, USA. ACM. DOI: 10.1145/2390848.2390861
Back to article
42Percival, G., & Tzanetakis, G. (2014). Streamlined tempo estimation based on autocorrelation and cross-correlation with pulses. IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP), 22(12), 1765–1776. DOI: 10.1109/TASLP.2014.2348916
Back to article
43Raffel, C. (2016). Learning-Based Methods for Comparing Sequences, with Applications to Audio-to-MIDI Alignment and Matching. PhD thesis, Columbia University, USA. DOI: 10.1109/ICASSP.2016.7471641
Back to article
44Raffel, C., McFee, B., Humphrey, E. J., Salamon, J., Nieto, O., Liang, D., & Ellis, D. P. W. (2014). mir_eval: A transparent implementation of common MIR metrics. In Proceedings of the 15th International Society for Music Information Retrieval Conference (ISMIR). Taipei, Taiwan.
Back to article
45Salamon, J. (2019). What’s broken in music informatics research? Three uncomfortable statements. In 36th International Conference on Machine Learning (ICML), Workshop on Machine Learning for Music Discovery. Long Beach, CA, USA.
Back to article
46Salamon, J., Gomez, E., Ellis, D. P., & Richard, G. (2014). Melody extraction from polyphonic music signals: Approaches, applications, and challenges. IEEE Signal Processing Magazine, 31(2), 118–134. DOI: 10.1109/MSP.2013.2271648
Back to article
47Salamon, J., & Urbano, J. (2012). Current challenges in the evaluation of predominant melody extraction algorithms. In Proceedings of the 13th International Society for Music Information Retrieval Conference (ISMIR), pages 289–294. Porto, Portugal.
Back to article
48Schedl, M., Flexer, A., & Urbano, J. (2013). The neglected user in music information retrieval research. Journal of Intelligent Information Systems, 41(3), 523–539. DOI: 10.1007/s10844-013-0247-6
Back to article
49Schedl, M., Zamani, H., Chen, C.-W., Deldjoo, Y., & Elahi, M. (2018). Current challenges and visions in music recommender systems research. International Journal of Multimedia Information Retrieval, 7(2), 95–116. DOI: 10.1007/s13735-018-0154-2
Back to article
50Scheirer, E. D. (1998). Tempo and beat analysis of acoustical musical signals. Journal of the Acoustical Society of America, 103(1), 588–601. DOI: 10.1121/1.421129
Back to article
51Schreiber, H., & Muller, M. (2014). Exploiting global features for tempo octave correction. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pages 639–643. Florence, Italy. DOI: 10.1109/ICASSP.2014.6853674
Back to article
52Schreiber, H., & Muller, M. (2017). A post-processing procedure for improving music tempo estimates using supervised learning. In Proceedings of the 18th International Society for Music Information Retrieval Conference (ISMIR), pages 235–242. Suzhou, China.
Back to article
53Schreiber, H., & Muller, M. (2018a). A crowd-sourced experiment for tempo estimation of electronic dance music. In Proceedings of the 19th International Society for Music Information Retrieval Conference (ISMIR). Paris, France.
Back to article
54Schreiber, H., & Muller, M. (2018b). A single-step approach to musical tempo estimation using a convolutional neural network. In Proceedings of the 19th International Society for Music Information Retrieval Conference (ISMIR). Paris, France.
Back to article
55Serra, X. (2014). Creating research corpora for the computational study of music: The case of the CompMusic project. In Proceedings of the AES International Conference on Semantic Audio. London, UK. Audio Engineering Society.
Back to article
56Serra, X., Magas, M., Benetos, E., Chudy, M., Dixon, S., Flexer, A., Gomez, E., Gouyon, F., Herrera, P., Jorda, S., Paytuvi, O., Peeters, G., Schluter, J., Vinet, H., & Widmer, G. (2013). Roadmap for Music Information ReSearch. http://mires.eecs.qmul.ac.uk/files/MIRES_Roadmap_ver_1.0.0.pdf
Back to article
57Slaney, M. (2011). Web-scale multimedia analysis: Does content matter? IEEEMultiMedia, 18(2), 12–15. DOI: 10.1109/MMUL.2011.34
Back to article
58Sturm, B. L. (2013a). Classification accuracy is not enough. Journal of Intelligent Information Systems, 41(3), 371–406. DOI: 10.1007/s10844-013-0250-y
Back to article
59Sturm, B. L. (2013b). The GTZAN dataset: Its contents, its faults, their effects on evaluation, and its future use. CoRR, abs/1306.1461.
Back to article
60Sturm, B. L. (2014). Faults in the ballroom dataset. Blog post http://media.aau.dk/null_space_pursuits/2014/01/ballroom-dataset.html, last accessed 4/29/2020.
Back to article
61Sturm, B. L. (2016). Revisiting priorities: Improving MIR evaluation practices. In Proceedings of the 17th International Society for Music Information Retrieval Conference (ISMIR). New York, NY, USA.
Back to article
62Sturm, B. L., Bardeli, R., Langlois, T., & Emiya, V. (2014). Formalizing the problem of music description. In Proceedings of the 15th International Society for Music Information Retrieval Conference (IS-MIR), pages 89–94. Taipei, Taiwan.
Back to article
63Tzanetakis, G., & Cook, P. (2002). Musical genre classification of audio signals. IEEE Transactions on Speech and Audio Processing, 10(5), 293–302. DOI: 10.1109/TSA.2002.800560
Back to article
64Tzanetakis, G., & Percival, G. (2013). An effective, simple tempo estimation method based on selfsimilarity and regularity. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP). Vancouver, Canada. DOI: 10.1109/ICASSP.2013.6637645
Back to article
65Urbano, J., Schedl, M., & Serra, X. (2013). Evaluation in music information retrieval. Journal of Intelligent Information Systems, 41(3), 345–369. DOI: 10.1007/s10844-013-0249-4
Back to article
66Vignoli, F., & Pauws, S. (2005). A music retrieval system based on user driven similarity and its evaluation. In Proceedings of the 6th International Conference on Music Information Retrieval (ISMIR), pages 272–279. London, UK.
Back to article
67Waterhouse, J., Hudson, P., & Edwards, B. (2010). Effects of music tempo upon submaximal cycling performance. Scandinavian Journal of Medicine & Science in Sports, 20(4), 662–669. DOI: 10.1111/j.1600-0838.2009.00948.x
Back to article
68Zapata, J. R., & Gomez, E. (2011). Comparative evaluation and combination of audio tempo estimation approaches. In 42nd AES Conference on Semantic Audio. Ilmenau, Germany.
Back to article

Music Tempo Estimation: Are We Done Yet?

Abstract

Paradigm

My account