On End-to-End White-Box Adversarial Attacks in Music Information Retrieval

Katharina Prinz; Arthur Flexer; Gerhard Widmer

doi:10.5334/tismir.85

Abstract

Small adversarial perturbations of input data can drastically change the performance of machine learning systems, thereby challenging their validity. We compare several adversarial attacks targeting an instrument classifier, where for the first time in Music Information Retrieval (MIR) the perturbations are computed directly on the waveform. The attacks can reduce the accuracy of the classifier significantly, while at the same time keeping perturbations almost imperceptible. Furthermore, we show the potential of adversarial attacks being a security issue in MIR by artificially boosting playcounts through an attack on a real-world music recommender system.

References

1Akhtar, N., and Mian, A. S. (2018). Threat of adversarial attacks on deep learning in computer vision: A survey. IEEE Access, 6: 14410–14430. DOI: 10.1109/ACCESS.2018.2807385
Back to article
2Bhalke, D. G., Rao, C. B. R., and Bormane, D. S. (2016). Automatic musical instrument classification using fractional Fourier transform based MFCC features and counter propagation neural network. Journal of Intelligent Information Systems, 46(3): 425–446. DOI: 10.1007/s10844-015-0360-9
Back to article
3Carlini, N., and Wagner, D. A. (2018). Audio adversarial examples: Targeted attacks on speech-to-text. In Proc. of the IEEE Security and Privacy Workshops, pages 1–7. IEEE. DOI: 10.1109/SPW.2018.00009
Back to article
4Deldjoo, Y., Noia, T. D., and Merra, F. A. (2021). A survey on adversarial recommender systems: From attack/defense strategies to generative adversarial networks. ACM Computing Surveys, 54(2): 1–38. DOI: 10.1145/3439729
Back to article
5Du, T., Ji, S., Li, J., Gu, Q., Wang, T., and Beyah, R. (2020). SirenAttack: Generating adversarial audio for end-to-end acoustic systems. In Proc. of the 15th ACM Asia Conference on Computer and Communications Security, pages 357–369. ACM. DOI: 10.1145/3320269.3384733
Back to article
6Engel, J. H., Hantrakul, L., Gu, C., and Roberts, A. (2020). DDSP: Differentiable digital signal processing. In Proc. of the 8th International Conference on Learning Representations.
Back to article
7Feldbauer, R., and Flexer, A. (2019). A comprehensive empirical comparison of hubness reduction in high-dimensional spaces. Knowledge and Information Systems, 59(1): 137–166. DOI: 10.1007/s10115-018-1205-y
Back to article
8Feldbauer, R., Rattei, T., and Flexer, A. (2020). scikithubness: Hubness reduction and approximate neighbor search. Journal of Open Source Software, 5(45): 1957. DOI: 10.21105/joss.01957
Back to article
9Flexer, A., Dörfler, M., Schlüter, J., and Grill, T. (2018). Hubness as a case of technical algorithmic bias in music recommendation. In Proc. of the IEEE International Conference on Data Mining Workshops, pages 1062–1069. IEEE. DOI: 10.1109/ICDMW.2018.00154
Back to article
10Flexer, A., and Stevens, J. (2018). Mutual proximity graphs for improved reachability in music recommendation. Journal of New Music Research, 47(1): 17–28. DOI: 10.1080/09298215.2017.1354891
Back to article
11Fonseca, E., Plakal, M., Font, F., Ellis, D. P., and Serra, X. (2019). Audio tagging with noisy labels and minimal supervision. In Proc. of the Detection and Classification of Acoustic Scenes and Events Workshop, pages 69–73. New York University. DOI: 10.33682/w13e-5v06
Back to article
12Gasser, M., and Flexer, A. (2009). FM4 Soundpark: Audio-based music recommendation in everyday use. In Proc. of the 6th Sound and Music Computing Conference, pages 23–25.
Back to article
13Goodfellow, I. J., Shlens, J., and Szegedy, C. (2015). Explaining and harnessing adversarial examples. In Proc. of the 3rd International Conference on Learning Representations.
Back to article
14Holzapfel, A., Sturm, B. L., and Coeckelbergh, M. (2018). Ethical dimensions of music information retrieval technology. Transactions of the International Society for Music Information Retrieval, 1(1): 44–55. DOI: 10.5334/tismir.13
Back to article
15Ioffe, S., and Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proc. of the 32nd International Conference on Machine Learning, pages 448–456.
Back to article
16Kereliuk, C., Sturm, B. L., and Larsen, J. (2015). Deep learning, audio adversaries, and music content analysis. In Proc. of the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, pages 1–5. IEEE. DOI: 10.1109/WASPAA.2015.7336950
Back to article
17Kingma, D. P., and Ba, J. (2015). Adam: A method for stochastic optimization. In Proc. of the 3rd International Conference on Learning Representations.
Back to article
18Knees, P., Schnitzer, D., and Flexer, A. (2014). Improving neighborhood-based collaborative filtering by reducing hubness. In Proc. of the 4th International Conference on Multimedia Retrieval, page 161. ACM. DOI: 10.1145/2578726.2578747
Back to article
19Lostanlen, V., Andén, J., and Lagrange, M. (2018). Extended playing techniques: The next milestone in musical instrument recognition. In Proc. of the 5th International Conference on Digital Libraries for Musicology, pages 1–10. ACM. DOI: 10.1145/3273024.3273036
Back to article
20Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Vladu, A. (2018). Towards deep learning models resistant to adversarial attacks. In Proc. of the 6th International Conference on Learning Representations.
Back to article
21Mandel, M. I., and Ellis, D. P. (2005). Song-level features and support vector machines for music classification. In Proc. of the 6th International Conference on Music Information Retrieval, pages 594–599.
Back to article
22Nair, V., and Hinton, G. E. (2010). Rectified linear units improve restricted Boltzmann machines. In Proc. of the 27th International Conference on Machine Learning, pages 807–814.
Back to article
23Pachet, F., and Aucouturier, J.-J. (2004). Improving timbre similarity: How high is the sky? Journal of Negative Results in Speech and Audio Sciences, 1(1): 1–13.
Back to article
24Paischer, F., Prinz, K., and Widmer, G. (2019). Audio tagging with convolutional neural networks trained with noisy data. In Proc. of the Detection and Classification of Acoustic Scenes and Events Workshop.
Back to article
25Qin, Y., Carlini, N., Cottrell, G. W., Goodfellow, I. J., and Raffel, C. (2019). Imperceptible, robust, and targeted adversarial examples for automatic speech recognition. In Proc. of the 36th International Conference on Machine Learning, pages 5231–5240.
Back to article
26Radovanović, M., Nanopoulos, A., and Ivanović, M. (2010). Hubs in space: Popular nearest neighbors in high-dimensional data. Journal of Machine Learning Research, 11(86): 2487–2531.
Back to article
27Rodríguez-Algarra, F., Sturm, B. L., and Dixon, S. (2019). Characterising confounding effects in music classification experiments through interventions. Transactions of the International Society for Music Information Retrieval, 2(1): 52–66. DOI: 10.5334/tismir.24
Back to article
28Schedl, M. (2019). Deep learning in music recommendation systems. Frontiers in Applied Mathematics and Statistics, 5: 44. DOI: 10.3389/fams.2019.00044
Back to article
29Schnitzer, D., Flexer, A., Schedl, M., and Widmer, G. (2012). Local and global scaling reduce hubs in space. Journal of Machine Learning Research, 13(1): 2871–2902.
Back to article
30Schönherr, L., Kohls, K., Zeiler, S., Holz, T., and Kolossa, D. (2019). Adversarial attacks against automatic speech recognition systems via psychoacoustic hiding. In Proc. of the 26th Annual Network and Distributed System Security Symposium. The Internet Society. DOI: 10.14722/ndss.2019.23288
Back to article
31Sturm, B. L. (2013). Classification accuracy is not enough: On the evaluation of music genre recognition systems. Journal of Intelligent Information Systems, 41(3): 371–406. DOI: 10.1007/s10844-013-0250-y
Back to article
32Sturm, B. L. (2014). A simple method to determine if a music information retrieval system is a “horse”. IEEE Transactions on Multimedia, 16(6): 1636–1644. DOI: 10.1109/TMM.2014.2330697
Back to article
33Sturm, B. L. (2016). The “horse” inside: Seeking causes behind the behaviors of music content analysis systems. Computers in Entertainment, 14(2): 1–32. DOI: 10.1145/2967507
Back to article
34Subramanian, V., Pankajakshan, A., Benetos, E., Xu, N., McDonald, S., and Sandler, M. (2020). A study on the transferability of adversarial attacks in sound event classification. In Proc. of the IEEE International Conference on Acoustics, Speech and Signal Processing, pages 301–305. IEEE. DOI: 10.1109/ICASSP40776.2020.9054445
Back to article
35Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I. J., and Fergus, R. (2014). Intriguing properties of neural networks. In Proc. of the 2nd International Conference on Learning Representations.
Back to article
36Trochim, W. M., and Donnelly, J. P. (2001). The Research Methods Knowledge Base. Atomic Dog Publishing, Cincinnati, 2nd edition.
Back to article
37Urbano, J., Schedl, M., and Serra, X. (2013). Evaluation in music information retrieval. Journal of Intelligent Information Systems, 41(3): 345–369. DOI: 10.1007/s10844-013-0249-4
Back to article
38Zhang, W. E., Sheng, Q. Z., Alhazmi, A. A., and Li, C. (2020). Adversarial attacks on deep-learning models in natural language processing: A survey. ACM Transactions on Intelligent Systems and Technology, 11(3): 24: 1–41. DOI: 10.1145/3374217
Back to article

On End-to-End White-Box Adversarial Attacks in Music Information Retrieval

Abstract

Paradigm

My account