Have a personal or library account? Click to login
ImproVision Equilibrium: Toward Multimodal Musical Human–Machine Interaction Cover

ImproVision Equilibrium: Toward Multimodal Musical Human–Machine Interaction

Open Access
|Nov 2025

References

  1. Assayag, G., Bloch, G., Chemillier, M., Cont, A., and Dubnov, S. (2006). Omax brothers: A dynamic topology of agents for improvization learning. In Proceedings of the 1st ACM Workshop on Audio and Music Computing Multimedia (pp. 125132). Santa Barbara, CA, USA: ACM.
  2. Assayag, G., and Dubnov, S. (2004). Using factor oracles for machine improvisation. Soft Computing, 8(9), 604610.
  3. Atkinson, M. (2020). Types, styles, and spaces of possibility: Phenomenology and musical improvisation. Gestalt Theory, 42(3), 253270.
  4. Bishop, L., and Goebl, W. (2018). Beating time: How ensemble musicians’ cueing gestures communicate beat position and tempo. Psychology of Music, 46(1), 84106.
  5. Boden, M. A. (2013). Creativity as a Neuroscientific Mystery. University of Sussex.
  6. Bonnasse‑Gahot, L. (2012). Prototype De Logiciel D’harmonisation/Arrangement à la Volée: Somax v0. Technical report, IRCAM.
  7. Borg, J. (2019). Prototype De Logiciel D’harmonisation/Arrangement à la Volée: Somax v0. STMS IRCAM & Aalborg University, Internal Student Report.
  8. Brackett, J. (2010). Some notes on John Zorn’s Cobra. American Music, 28(1), 4475.
  9. Caillois, R. (2001). Man, Play, and Games. University of Illinois Press.
  10. Cassidy, G. G., and Paisley, A. M. (2013). Music‑games: A case study of their impact. Research Studies in Music Education, 35(1), 119138.
  11. Clayton, M., Sager, R., and Will, U. (2005). In time with the music: The concept of entrainment and its significance for ethnomusicology. European Meetings in Ethnomusicology, 11, 182.
  12. Davis, N. (2013). Human‑computer co‑creativity: Blending human and computational creativity. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (Vol. 9, pp. 912). Boston, Massachusetts, USA.
  13. Denis, G., and Jouvelot, P. (2005). Motivation‑driven educational game design: Applying best practices to music education. In Proceedings of the 2005 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology (pp. 462465). Valencia, Spain.
  14. Doelling, K. B., and Poeppel, D. (2015). Cortical entrainment to music and its modulation by expertise. Proceedings of the National Academy of Sciences, 112(45), E6233E6242.
  15. Dubnov, S., Assayag, G., and Gokul, V. (2022). Creative improvised interaction with generative musical systems. In 2022 IEEE 5th International Conference on Multimedia Information Processing and Retrieval (MIPR) (pp. 121126). California, USA. IEEE.
  16. Dubnov, S., Gokul, V., and Assayag, G. (2023). Switching machine improvisation models by latent transfer entropy criteria. Physical Sciences Forum, 5(1), 49.
  17. Fan, Z., Zhu, Y., He, Y., Sun, Q., Liu, H., and He, J. (2022). Deep learning on monocular object pose detection and tracking: A comprehensive overview. ACM Computing Surveys, 55(4), 140.
  18. Gordy, C. A. (1999). The Duo Piano Experience: Nonverbal Communication Between Ensemble Musicians (Doctoral dissertation). Concordia University.
  19. Greer, R., Fleig, L., and Dubnov, S. (2024). Creativity and visual communication from machine to musician: Sharing a score through a robotic camera. In 13th EAI International Conference ArtsIT, NYU, Abu Dhabi, UAE.
  20. Gritten, A., and King, E. (2006). Music and Gesture. Ashgate Publishing, Ltd.
  21. Gritten, A., and King, E. (2011). New Perspectives on Music and Gesture. Ashgate Publishing, Ltd.
  22. Hall, L. M., Buechler, C., Marusca, G., Brennan, S., and Levine, D. L. (2021). Utilizing chamber music to teach non‑verbal communication to medical students: A pilot initiative. Cureus, 13(4), e14587.
  23. Harrop‑Allin, S. (2017). Multimodality and the multiliteracies pedagogy: “Design” and “recruitment” in South African children’s musical games. Journal of Research in Music Education, 65(1), 2551.
  24. Hedges, S. A. (1978). Dice music in the eighteenth century. Music & Letters, 59(2), 180187.
  25. Hsu, W., and Sosnick, M. (2009). Evaluating interactive music systems: An HCI approach. In Proceedings of the International Conference on New Interfaces for Musical Expression (pp. 2528). Pittsburgh, PA, USA.
  26. Huizinga, J. (1971). Homo Ludens: A Study of the Play‑Element in Culture. Beacon Press.
  27. Jimenez, A. P. M. (2018). Game co‑creation with young music students: A pedagogic approach to promote creativity and engagement. In European Conference on Games Based Learning (pp. 792799). Sophia Antipolis, France. Academic Conferences International Limited.
  28. Jin, S., Xu, L., Xu, J., Wang, C., Liu, W., Qian, C., Ouyang, W., and Luo, P. (2020). Whole‑body human pose estimation in the wild. In Proceedings of the European Conference on Computer Vision (ECCV).
  29. Jordanous, A. (2012). A standardised procedure for evaluating creative systems: Computational creativity evaluation based on what it is to be creative. Cognitive Computation, 4, 246279.
  30. Kawase, S. (2014). Gazing behavior and coordination during piano duo performance. Attention, Perception, & Psychophysics, 76, 527540.
  31. Kim, J. H., Reifgerst, A., and Rizzonelli, M. (2019). Musical social entrainment. Music & Science, 2, 2059204319848991.
  32. King E. (2013). Musical shaping gestures: Considerations about terminology and methodology. Empirical Musicology Review, 6871. 10.18061/emr.v8i1.3925.
  33. Kurkul, W. W. (2007). Nonverbal communication in one‑to‑one music performance instruction. Psychology of Music, 35(2), 327362.
  34. Luck, G., and Nte, S. (2008). An investigation of conductors’ temporal gestures and conductor‑musician synchronization, and a first experiment. Psychology of Music, 36(1), 8199.
  35. Luck, G., and Sloboda, J. (2008). Exploring the spatio‑temporal properties of simple conducting gestures using a synchronization task. Music Perception, 25(3), 225239.
  36. Manzo, V. J. (2016). Max/MSP/Jitter for Music: A Practical Guide to Developing Interactive Music Systems for Education and More. Oxford University Press.
  37. Pearson, L., and Pouw, W. (2022). Gesture‑vocal coupling in Karnatak music performance: A neuro‑bodily distributed aesthetic entanglement. Annals of the New York Academy of Sciences, 1515(1), 219236.
  38. Qian, R., Hu, D., Dinkel, H., Wu, M., Xu, N., and Lin, W. (2020). Multiple sound sources localization from coarse to fine. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16. (pp. 292308). Springer.
  39. Rascon, C., and Meza, I. (2017). Localization of sound sources in robotics: A review. Robotics and Autonomous Systems, 96, 184210.
  40. Ross, J. M., and Balasubramaniam, R. (2022). Time perception for musical rhythms: Sensorimotor perspectives on entrainment, simulation, and prediction. Frontiers in Integrative Neuroscience, 16, 916220.
  41. Rowe, R. (1992). Interactive Music Systems: Machine Listening and Composing. MIT Press.
  42. Seddon, F. A. (2005). Modes of communication during jazz improvisation. British Journal of Music Education, 22(1), 4761.
  43. Senocak, A., Ryu, H., Kim, J., and Kweon, I. S. (2022). Less can be more: Sound source localization with a classification model. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (pp. 33083317). Waikoloa, Hawaii, USA.
  44. Stanley, T. T. (2009). Butch Morris and the art of conduction. University of Maryland.
  45. Tekinbas, K. S., and Zimmerman, E. (2003). Rules of Play: Game Design Fundamentals. MIT Press.
  46. Thompson, W. (2024). Soundpainting. https://www.soundpainting.com/soundpainting/.
  47. Tymoczko, D. (2008). Scale theory, serial theory and voice leading. Music Analysis, 27(1), 149.
  48. van der Schyff, D. (2013). The free improvisation game: Performing John Zorn’s Cobra. Journal of Research in Music Performance, 2013, 111.
DOI: https://doi.org/10.5334/tismir.225 | Journal eISSN: 2514-3298
Language: English
Submitted on: Sep 2, 2024
|
Accepted on: Oct 6, 2025
|
Published on: Nov 17, 2025
Published by: Ubiquity Press
In partnership with: Paradigm Publishing Services
Publication frequency: 1 issue per year

© 2025 Ross Greer, Laura Fleig, Shlomo Dubnov, published by Ubiquity Press
This work is licensed under the Creative Commons Attribution 4.0 License.