Music Information Retrieval and Contemporary Classical Music: A Successful Failure

Carmine-Emanuele Cella

doi:10.5334/tismir.55

Abstract

This paper is about the story of my relationship, as a contemporary music composer, with computational tools that are situated in the areas of signal processing, machine learning and music information retrieval (MIR). I believe that sharing this story can be useful to the MIR community since it illustrates the problems that can arise when you try to use these techniques in the context of contemporary music creation. Since this is a personal story, I will refer to experiences that I had during about fifteen years of usage of MIR-related technologies. I will show how these technologies tried to (unsuccessfully) shape my musical thinking and why I believe that some of them have come to an end. Finally, I will propose new possible directions for the future of MIR.

References

1Andén, J., & Mallat, S. (2014). Deep scattering spectrum. IEEE Transactions on Signal Processing, 62(16), 4114–4128. DOI: 10.1109/TSP.2014.2326991
Back to article
2Andersen, K., & Knees, P. (2016). Conversations with expert users in music retrieval and research challenges for creative MIR. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR), pages 122–128.
Back to article
3Assayag, G., Rueda, C., Laurson, M., Agon, C., & Delerue, O. (1999). Computer-assisted composition at IRCAM: From PatchWork to OpenMusic. Computer Music Journal, 23(3), 59–72. DOI: 10.1162/014892699559896
Back to article
4Boulanger, R. C., editor. (2000). The Csound Book: Perspectives in Software Synthesis, Sound Design, Signal Processing, and Programming. MIT Press.
Back to article
5Briot, J.-P., Hadjeres, G., & Pachet, F. (2019). Deep Learning Techniques for Music Generation. Computational Synthesis and Creative Systems Series. Springer Verlag. DOI: 10.1007/978-3-319-70163-9
Back to article
6Bruna, J., & Mallat, S. (2013). Invariant scattering convolution networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8), 1872–1886. DOI: 10.1109/TPAMI.2012.230
Back to article
7Burred, J. J., Cella, C.-E., Peeters, G., Roebel, A., & Schwarz, D. (2008). Using the SDIF sound description interchange format for audio features. In Proceedings of the International Conference on Music Information Retrieval (ISMIR), pages 427–432.
Back to article
8Cardoso, A., Veale, T., & Wiggins, G. A. (2009). Converging on the divergent: The history (and future) of the International Joint Workshops in Computational Creativity. AI Magazine, 30(3), 15–22. DOI: 10.1609/aimag.v30i3.2252
Back to article
9Carpentier, G., Tardieu, D., Assayag, G., & Saint-James, E. (2007). An evolutionary approach to computer-aided orchestration. In M. Giacobini, editors, Applications of Evolutionary Computing: EvoWorkshops 2007, volume 4448, pages 488–497. Springer. DOI: 10.1007/978-3-540-71805-5_54
Back to article
10Carpentier, G., Tardieu, D., Harvey, J., Assayag, G., & Saint-James, E. (2010). Predicting timbre features of instrument sound combinations: Application to automatic orchestration. Journal of New Music Research, 39(1), 47–61. DOI: 10.1080/09298210903581566
Back to article
11Cella, C.-E. (2011a). On symbolic representations of music. PhD thesis, University of Bologna.
Back to article
12Cella, C.-E. (2011b). Sound-types: A new framework for symbolic sound analysis and synthesis. In Proceedings of the International Computer Music Conference (ICMC), pages 179–184.
Back to article
13Cella, C.-E. (2017). Machine listening intelligence. In Proceedings of the International Workshop on Deep Learning for Music, pages 50–55.
Back to article
14Cella, C.-E., & Burred, J. J. (2013). Advanced sound hybridizations by means of the theory of soundtypes. In Proceedings of the International Computer Music Conference (ICMC), pages 39–46.
Back to article
15Cella, C.-E., & Esling, P. (2018). Open-source modular toolbox for computer-aided orchestration. In Proceedings of Timbre 2018: Timbre is a Many-Splendored Thing, pages 93–94.
Back to article
16Choi, K., Fazekas, G., Sandler, M., & Cho, K. (2017). Convolutional recurrent neural networks for music classification. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2392–2396. DOI: 10.1109/ICASSP.2017.7952585
Back to article
17Crayencour, H.-C., & Cella, C.-E. (2019). Learning, probability and logic: Toward a unified approach for content-based music information retrieval. Frontiers in Digital Humanities, 6(6). DOI: 10.3389/fdigh.2019.00006
Back to article
18Davis, S. B., & Mermelstein, P. (1980). Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences. IEEE Transactions on Acoustics, Speech, and Signal Processing, 28(4), 357–366. DOI: 10.1109/TASSP.1980.1163420
Back to article
19Deserno, S. (2015). Algorithmic composition: An overview of the field, inspired by a criticism of its methods. Seminar topics in computer music, RWTH Aachen University.
Back to article
20Dieleman, S., & Schrauwen, B. (2014). End-to-end learning for music audio. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6964–6968. DOI: 10.1109/ICASSP.2014.6854950
Back to article
21Donin, N., & Feneyrou, L., editors. (2017). Théories de la composition musicale au xxe siècle. Symétrie.
Back to article
22Fernández, J. D., & Vico, F. (2013). AI methods in algorithmic composition: A comprehensive survey. Journal of Artificial Intelligence Research, 48(1), 513–582. DOI: 10.1613/jair.3908
Back to article
23Gabrielli, L., Cella, C.-E., Vesperini, F., Droghini, D., Principi, E., & Squartini, S. (2018). Deep learning for timbre modification and transfer: An evaluation study. In Proceedings of the Audio Engineering Society (AES) Convention 144.
Back to article
24Ghisi, D. (2017). Music Across Music: Towards a Corpus-Based, Interactive Computer-Aided Composition. PhD thesis, IRCAM.
Back to article
25Gillick, J., Cella, C.-E., & Bamman, D. (2019). Estimating unobserved audio features for target-based orchestration. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR), pages 192–199.
Back to article
26He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778. DOI: 10.1109/CVPR.2016.90
Back to article
27Hirn, M., Mallat, S., & Poilvert, N. (2017). Wavelet scattering regression of quantum chemical energies. Journal of Multiscale Modeling Simulation, 15(2), 827–863. DOI: 10.1137/16M1075454
Back to article
28Humphrey, E. J., Bello, J. P., & LeCun, Y. (2015). Feature learning and deep architectures: New directions for music informatics. Journal of Intelligent Information Systems, 41(3), 461–481. DOI: 10.1007/s10844-013-0248-5
Back to article
29Humphrey, E. J., Turnbull, D., & Collins, T. (2013). A brief review of creative MIR. In International Society for Music Information Retrieval Conference (ISMIR), Late-Breaking News and Demos.
Back to article
30Klien, V., Grill, T., & Flexer, A. (2012). On automated annotation of acousmatic music. Journal of New Music Research, 41(2), 153–173. DOI: 10.1080/09298215.2011.618226
Back to article
31Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Proceedings of the International Conference on Neural Information Processing Systems (NIPS), pages 1097–1105.
Back to article
32Lacoste, A., & Eck, D. (2005). Onset detection with artificial neural networks. In Music Information Retrieval Evaluation eXchange (MIREX), pages 1097–1105.
Back to article
33LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521, 436–444. DOI: 10.1038/nature14539
Back to article
34Lipp, C. (1996). Real-time interactive digital signal processing: A view of computer music. Computer Music Journal, 20(4), 21–24. DOI: 10.2307/3680412
Back to article
35Lostanlen, V., & Cella, C.-E. (2016). Deep convolutional networks on the pitch spiral for music instrument recognition. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR), pages 612–618.
Back to article
36Mallat, S. (2012). Group invariant scattering. Communications on Pure and Applied Mathematics, 65(10), 1331–1398. DOI: 10.1002/cpa.21413
Back to article
37Mallat, S. (2016). Understanding deep convolutional networks. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 374(2065).
Back to article
38Maresz, Y. (2013). On computer-assisted orchestration. Contemporary Music Review, 32(1), 99–109. DOI: 10.1098/rsta.2015.0203
Back to article
39McAdams, S. (1999). Perspectives on the contribution of timbre to musical structure. Computer Music Journal, 23(3), 85–102. DOI: 10.1080/07494467.2013.774515
Back to article
40McAdams, S., & Giordano, B. L. (2016). The perception of musical timbre. In S. Hallam, I. Cross & M. Thaut, editors, The Oxford Handbook of Music Psychology (2nd ed.). Oxford University Press. DOI: 10.1162/014892699559797
Back to article
41Mehri, S., Kumar, K., Gulrajani, I., Kumar, R., Jain, S., Sotelo, J., Courville, A., & Bengio, Y. (2015). SampleRNN: An unconditional end-to-end neural audio generation model. In Proceedings of the International Conference on Learning Representations (ICLR). DOI: 10.1093/oxfordhb/9780198722946.013.12
Back to article
42Mermelstein, P. (1976). Distance measures for speech recognition, psychological and instrumental. Pattern Recognition and Artificial Intelligence, 116, 374–388.
Back to article
43Müller, M. (2015). Fundamentals of Music Processing: Audio, Analysis, Algorithms, Applications. Springer.
Back to article
44Nilsson, N. J. (2009). The Quest for Artificial Intelligence. Cambridge University Press. DOI: 10.1017/CBO9780511819346
Back to article
45Papadopoulos, H., & Peeters, G. (2007). Large-scale study of chord estimation algorithms based on chroma representation and HMM. In Proceedings of the IEEE International Workshop on Content-Based Multimedia Indexing (CBMI), pages 53–60. DOI: 10.1109/CBMI.2007.385392
Back to article
46Papadopoulos, H., & Peeters, G. (2011). Joint estimation of chords and downbeats. IEEE Transactions on Audio, Speech, and Language Processing, 19(1), 138–152. DOI: 10.1109/TASL.2010.2045236
Back to article
47Peeters, G. (2004). A large set of audio features for sound description (similarity and classification) in the CUIDADO project. Technical report, IRCAM.
Back to article
48Sak, H., Senior, A., & Beaufays, F. (2014). Long short-term memory recurrent neural network architectures for large scale acoustic modeling. In Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH), pages 338–342.
Back to article
49Schlüter, J., & Böck, S. (2014). Improved musical onset detection with convolutional neural networks. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6979–6983. DOI: 10.1109/ICASSP.2014.6854953
Back to article
50Sifre, L., & Mallat, S. (2013). Rotation, scaling and deformation invariant scattering for texture discrimination. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1233–1240. DOI: 10.1109/CVPR.2013.163
Back to article
51Smalley, D. (1997). Spectromorphology: Explaining sound-shapes. Organised Sound, 2(2), 107–126. DOI: 10.1017/S1355771897009059
Back to article
52Srinivas, S., Sarvadevabhatla, R. K., Mopuri, K. R., Prabhu, N., Kruthiventi, S. S. S., & Babu, R. V. (2016). A taxonomy of deep convolutional neural nets for computer vision. Frontiers in Robotics and AI, 2, 36. DOI: 10.3389/frobt.2015.00036
Back to article
53van den Oord, A., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016). WaveNet: A generative model for raw audio. arXiv preprint arXiv:1609.03499.
Back to article
54Vinet, H. (2003). The representation level of music information. In Proceedings of the International Symposium on Computer Music Modeling and Retrieval (CMMR), pages 193–209. DOI: 10.1007/978-3-540-39900-1_17
Back to article
55Vinet, H. (2008). Science and technology of music and sound: The IRCAM roadmap. Journal of New Music Research, 36(3), 207–226. DOI: 10.1080/09298210701859313
Back to article
56Wiggins, G. A. (2009). Semantic gap?? Schemantic schmap!! Methodological considerations in the scientific study of music. In Proceedings of the IEEE International Symposium on Multimedia, pages 477–482. DOI: 10.1109/ISM.2009.36
Back to article
57Wiggins, G. A., Pearce, M. T., & Müllensiefen, D. (2009). Computational modelling of music cognition and musical creativity. In R. T. Dean, editor. The Oxford Handbook of Computer Music, pages 383–420. Oxford University Press.
Back to article
58Ycart, A., & Benetos, E. (2017). A study on LSTM networks for polyphonic music sequence modelling. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR), pages 421–427.
Back to article

Music Information Retrieval and Contemporary Classical Music: A Successful Failure

Abstract

Paradigm

My account