A Review of String Instrument Synthesis Methods for Use in Interactive Systems
References
- Adrien, J.‑M. (1991).
The missing link: Modal synthesis . In G. De Poli, A. Piccialli, and C. Roads (Eds.), Representations of Musical Signals (pp. 269–298). MIT Press. - Arksey, H., and O’Malley, L. (2005). Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology, 8(1), 19–32.
- Baines, A. (1992). The Oxford Companion to Musical Instruments. Oxford University Press.
- Bank, B., Zambon, S., and Fontana, F. (2010). A modal‑based real‑time piano synthesizer. IEEE Transactions on Speech and Audio Processing, 18(4), 809–821.
- Baoueb, T., Bie, X., Janati, H., and Richard, G. (2024).
WaveTransfer: A flexible end‑to‑end multi‑instrument timbre transfer with diffusion . In 34th IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2024 (pp. 1–6). IEEE. - Bensa, J., Bilbao, S., Kronland‑Martinet, R., and Smith, J. O. (2003). The simulation of piano string vibration: From physical models to finite difference schemes and digital waveguides. The Journal of the Acoustical Society of America, 114(2), 1095–1107.
- Bilbao, S. (2009). Numerical sound synthesis: Finite difference schemes and simulation in musical acoustics. Wiley Publishing.
- Bitton, A., Esling, P., and Harada, T. (2020). Neural granular sound synthesis. CoRR, abs/2008.01393.
- Cadoz, C., Luciani, A., and Florens, J. L. (1993). CORDIS‑ANIMA: A modeling and simulation system for sound and image synthesis–The general formalism. Computer Music Journal, 17(1), 19–29.
- Caillon, A., and Esling, P. (2021). RAVE: A variational autoencoder for fast and high‑quality neural audio synthesis. CoRR, abs/2111.05011.
- Carney, M., Li, C., Toh, E., Zada, N., Yu, P., and Engel, J. H. (2021). Tone transfer: In‑browser interactive neural audio synthesis. In Joint Proceedings of the ACM IUI 2021 Workshops Co‑Located with 26th ACM Conference on Intelligent User Interfaces (ACM IUI 2021), of CEUR Workshop Proceedings, 2903.
CEUR-WS.org . - Carrillo, A. P., and Bonada, J. (2010). The bowed tube: A virtual violin. In Proceedings of the International Conference on New Interfaces for Musical Expression (pp. 229–232). Sydney, Australia.
- Caspe, F., McPherson, A., and Sandler, M. (2022). DDX7: differentiable FM synthesis of musical instrument sounds. In Proceedings of the 23rd International Society for Music Information Retrieval Conference, ISMIR 2022 (pp. 608–616).
- Castagné, N., and Cadoz, C. (2003).
10 criteria for evaluating physical modelling schemes for music creation . In Proceedings of the 6th Digital Audio Effects (pp. 1–7). London, United Kingdom. - Chowning, J. (1973). The synthesis of complex audio spectra by means of frequency modulation. Journal of the Audio Engineering Society, 21(7), 526–534.
- Dai, Y. R., Yang, H. C., and Su, A. W. (2021).
Efficient synthesis of violin sounds using a BiLSTM network based source filter model . In 150th Audio Engineering Society Convention, AES 2021, 150th Audio Engineering Society Convention, AES 2021. Audio Engineering Society. - Défossez, A., Zeghidour, N., Usunier, N., Bottou, L., and Bach, F. R. (2018). SING: Symbol‑to‑instrument neural generator. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018 (pp. 9055–9065).
- Ding, Y., and Gerhard, D. (2004). Analysis and synthesis of the guqin ‑A Chinese traditional instrument. Canadian Acoustics, 32(3), 122–123.
- Donahue, C., McAuley, J. J., and Puckette, M. S. (2019). Adversarial audio synthesis. In 7th International Conference on Learning Representations, ICLR 2019. New Orleans, LA, USA.
OpenReview.net . - Dong, H.‑W., Hsiao, W.‑Y., Yang, L.‑C., and Yang, Y.‑H. (2018). MuseGAN: Multi‑track sequential generative adversarial networks for symbolic music generation and accompaniment. In Proceedings of the Thirty‑Second AAAI Conference on Artificial Intelligence (AAAI‑18), the 30th Innovative Applications of Artificial Intelligence (IAAI‑18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI‑18) (pp. 34–41). AAAI Press.
- Engel, J. H., Agrawal, K. K., Chen, S., Gulrajani, I., Donahue, C., and Roberts, A. (2019). GANSynth: Adversarial neural audio synthesis. In 7th International Conference on Learning Representations, ICLR 2019.
OpenReview.net . - Engel, J. H., Hantrakul, L., Gu, C., and Roberts, A. (2020). DDSP: Differentiable digital signal processing. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia. April 26–30, 2020.
OpenReview.net . - Engel, J. H., Resnick, C., Roberts, A., Dieleman, S., Norouzi, M., Eck, D., and Simonyan, K. (2017). Neural audio synthesis of musical notes with WaveNet autoencoders. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Volume 70 of Proceedings of Machine Learning Research (pp. 1068–1077). PMLR.
- Erkut, C., Karjalainen, M., Huang, P., and Välimäki, V. (2002). Acoustical analysis and model‑based sound synthesis of the kantele. Journal of the Acoustical Society of America, 112(4), 1681–1691.
- Erkut, C., Laurson, M., Kuuskankare, M., and Välimäki, V. (2001). Model‑based synthesis of the ud and the renaissance lute. In Proceedings of the 2001 International Computer Music Conference, ICMC 2001. Michigan Publishing.
- Fletcher, N. H., and Rossing, T. D. (1998). The Physics of Musical Instruments. Springer.
- Florens, J.‑L. (2003).
Expressive bowing on a virtual string instrument . In Gesture‑Based Communication in Human‑Computer Interaction, 5th International Gesture Workshop, GW 2003, Volume 2915 of Lecture Notes in Computer Science (pp. 487–496). Springer. - Fraisse, V., Wanderley, M. M., and Guastavino, C. (2021). Comprehensive framework for describing interactive sound installations: Highlighting trends through a systematic review. Multimodal Technologies and Interaction, 5(4), 19.
- Ganis, F., Knudsen, E. F., Lyster, S. V. K., Otterbein, R., Südholt, D., and Erkut, C. (2021). Real‑time timbre transfer and sound synthesis using DDSP. In SMC 2021 ‑ Proceedings of the 18th Sound and Music Computing Conference (Vol. 2021, pp. 175–182). Virtual Sound and Music Computing Network.
- Gehlhaar, R., Girao, L. M., and Rodrigues, P. M. (2011). CaDaReMi. An educational interactive music game. International Journal on Disability and Human Development, 10(1), 25–29.
- Gómez‑Sirvent, J. L., López de la Rosa, F., Sánchez‑Reolid, R., Herrera, R. M., and Fernández‑Caballero, A. (2025). Musical instruments in extended reality: A systematic review. International Journal of Human–Computer Interaction, 41(16), 10143–10162.
- Hawley, S. H. (2020). Synthesis of musical instrument sounds: Physics‑based modeling or machine learning? Acoustics Today, 16(1), 20.
- Hawthorne, C., Simon, I., Roberts, A., Zeghidour, N., Gardner, J., Manilow, E., and Engel, J. H. (2022). Multi‑instrument music synthesis with spectrogram diffusion. In Proceedings of the 23rd International Society for Music Information Retrieval Conference, ISMIR 2022 (pp. 598–607).
- Hawthorne, C., Stasyuk, A., Roberts, A., Simon, I., Huang, C.‑Z. A., Dieleman, S., Elsen, E., Engel, J. H., and Eck, D. (2019). Enabling factorized piano music modeling and generation with the MAESTRO dataset. In 7th International Conference on Learning Representations, ICLR 2019.
OpenReview.net . - Hayes, B., Saitis, C., and Fazekas, G. (2021). Neural waveshaping synthesis. In Proceedings of the 22nd International Society for Music Information Retrieval Conference (pp. 9). ISMIR.
- Hayes, B., Shier, J., Fazekas, G., McPherson, A., and Saitis, C. (2024). A review of differentiable digital signal processing for music and speech synthesis. Frontiers in Signal Processing, 3, 2023.
- Hiller, L., and Ruiz, P. (1971). Synthesizing musical sounds by solving the wave equation for vibrating objects: Part 1. Journal of the Audio Engineering Society, 19(6), 462–470.
- Horner, A., Beauchamp, J., and Haken, L. (1993). Methods for multiple wavetable synthesis of musical instrument tones. AES: Journal of the Audio Engineering Society, 41(5), 336–356.
- Huang, Q., Park, D. S., Wang, T., Denk, T. I., Ly, A., Chen, N., Zhang, Z., Zhang, Z., Yu, J., Frank, C. H., Engel, J. H., Le, Q. V., Chan, W., and Han, W. (2023). Noise2Music: Text‑ conditioned music generation with diffusion models. CoRR, abs/2302.03917.
- Hung, T.‑Y., Wu, J.‑T., Kuo, Y.‑C., Hsiao, Y.‑W., Lin, T.‑W., and Su, L. (2024). A study on synthesizing expressive violin performances: Approaches and comparisons. CoRR, abs/2406.18089.
- Jack, R. H., Stockman, T., and McPherson, A. P. (2016).
Effect of latency on performer interaction and subjective quality assessment of a digital musical instrument . In Proceedings of the Audio Mostly 2016 (pp. 116–123). ACM. - Jaffe, D. A. (1995). Ten criteria for evaluating synthesis techniques. Computer Music Journal, 19(1), 76–87.
- Jonason, N., Sturm, B. L. T., and Thomé, C. (2020). The control‑synthesis approach for making expressive and controllable neural music synthesizers. In Proceedings of the 1st Joint Conference on AI Music Creativity (pp. 9). AIMC.
- Jonason, N., Wang, X., Cooper, E., Juvela, L., Sturm, B. L. T., and Yamagishi, J. (2023). DDSP‑based neural waveform synthesis of polyphonic guitar performance from string‑wise MIDI input. CoRR, abs/2309.07658.
- Kalchbrenner, N., Elsen, E., Simonyan, K., Noury, S., Casagrande, N., Lockhart, E., Stimberg, F., van den Oord, A., Dieleman, S., and Kavukcuoglu, K. (2018). Efficient neural audio synthesis. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Volume 80 of Proceedings of Machine Learning Research (pp. 2415–2424). PMLR.
- Karplus, K., and Strong, A. (1983). Digital synthesis of plucked‑ string and drum timbres. Computer Music Journal, 7(2), 43–55.
- Kim, D., Dong, H.‑W., and Jeong, D. (2025). ViolinDiff: Enhancing expressive violin synthesis with pitch bend conditioning. In 2025 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2025 (pp. 1–5). IEEE.
- Kim, H., Choi, S., and Nam, J. (2024). Expressive acoustic guitar sound synthesis with an instrument‑specific input representation and diffusion outpainting. In IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2024 (pp. 7620–7624). IEEE.
- Kumar, K., Kumar, R., de Boissiere, T., Gestin, L., Teoh, W. Z., Sotelo, J., de Brébisson, A., Bengio, Y., and Courville, A. C. (2019). MelGAN: Generative adversarial networks for conditional waveform synthesis. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019 (pp. 14881–14892).
- Lee, J. W., Park, J., Choi, M. J., and Lee, K. (2024, NeurIPS). Differentiable modal synthesis for physical modeling of planar string sound and motion simulation. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024.
- Leonard, J., and Villeneuve, J. (2019). MI‑GEN~: An efficient and accessible mass‑interaction sound synthesis toolbox. In Proceedings of the 16th Sound and Music Computing Conference (SMC), Malaga, Spain.
- Levitin, D. J., McAdams, S., and Adams, R. L. (2002). Control parameters for musical instruments: A foundation for new mappings of gesture to sound. Organised Sound, 7(2), 171–189.
- Liu, Q. (2024). An FM‑wavetable‑synthesized violin with natural vibrato and bow pressure. In Proceedings of the 2023 International Conference on Data Science, Advanced Algorithm and Intelligent Computing (DAI 2023) (pp. 243–250). Atlantis Press.
- Maman, B., Zeitler, J., Müller, M., and Bermano, A. H. (2024). Performance conditioning for diffusion‑based multi‑ instrument music synthesis. In IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2024 (pp. 5045–5049). IEEE.
- Maman, B., Zeitler, J., Müller, M., and Bermano, A. H. (2025). Multi‑aspect conditioning for diffusion‑based music synthesis: Enhancing realism and acoustic control. IEEE Transactions on Audio, Speech and Language Processing, 33, 68–81.
- Mathews, M. V. (1963). The digital computer as a musical instrument. Science, 142(3592), 553–557.
- Mathews, M. V., Miller, J. E., Moore, F. R., Pierce, J. R., and Risset, J. C. (1969). The technology of computer music. The MIT Press.
- Miranda, E. R., and Wanderley, M. (2006). New digital musical instruments: Control and interaction beyond the keyboard (computer music and digital audio series). A‑R Editions, Inc.
- Moog, R. A. (1965). Voltage controlled electronic music modules. Journal of the Audio Engineering Society, 13(3), 200–206.
- Narita, G., Shimizu, J., and Akama, T. (2023). GANStrument: Adversarial instrument sound synthesis with pitch‑ invariant instance conditioning. In IEEE International Conference on Acoustics, Speech and Signal Processing ICASSP 2023 (pp. 1–5). IEEE.
- Natsiou, A., and O’Leary, S. (2021). Audio representations for deep learning in sound synthesis: A review. In 18th IEEE/ACS International Conference on Computer Systems and Applications, AICCSA 2021, Tangier, Morocco, November 30–December 3, 2021 (pp. 1–8). IEEE.
- Nichols, C. (2002). The vBow: A virtual violin bow controller for mapping gesture to synthesis with haptic feedback. Organised Sound, 7, 215–220.
- Passalenti, A., Paisa, R., Nilsson, N. C., Andersson, N. S., Fontana, F., Nordahl, R., and Serafin, S. (2019). No strings attached: Force and vibrotactile feedback in a virtual guitar simulation. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (pp. 1116–1117).
- Puckette, M. S. (1996). Pure Data: Another integrated computer music environment. In Proceedings of the Second Intercollege Computer Music Concerts (pp. 37–41).
- Renault, L., Mignot, R., and Roebel, A. (2022). Differentiable piano model for MIDI‑to‑audio performance synthesis. In 25th International Conference on Digital Audio Effects (DAFx20in22), Vienna, Austria.
- Richardson, N., Kothapalli, S., Onteddu, A., Kundavaram, R., and Talla, R. R. (2023). AI‑driven optimization techniques for evolving software architecture in complex systems. ABC Journal of Advanced Research, 12, 71–84.
- Risset, J., and Mathews, M. V. (1969). Analysis of musical‑ instrument tones. Physics Today, 22(2), 23–30.
- Roberts, R. A., and Mullis, C. T. (1987). Digital signal processing. Addison‑Wesley Longman Publishing Co., Inc.
- Rossing, T. D (Ed.). (2010). The science of string instruments. Springer.
- Schwarz, D. (2007). Corpus‑based concatenative synthesis. IEEE Signal Processing Magazine, 24(2), 92–104.
- Selfridge, R., Moffat, D., Reiss, J., and Avital, E. (2024). Real‑time physical model of an aeolian harp. In 24th International Conference on Sound and Vibration 2017. London, UK: Institute of Acoustics.
- Serafin, S., Erkut, C., Kojs, J., Nordahl, R., and Nilsson, N. C. (2016).
Virtual reality musical instruments: Guidelines for multisensory interaction design . In Proceedings of the Audio Mostly 2016 (pp. 266–271). ACM. - Serra, X., and Smith, J. (1990). Spectral modeling synthesis: A sound analysis/synthesis system based on a deterministic plus stochastic decomposition. Computer Music Journal, 14(4), 12.
- Shan, S., Hantrakul, L., Chen, J., Avent, M., and Trevelyan, D. (2022). Differentiable wavetable synthesis. In IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2022 (pp. 4598–4602). IEEE.
- Siao, Y. S., Chang, W. L., and Su, A. W. (2006). Analysis and transynthesis of solo erhu recordings using additive/subtractive synthesis: 120th Convention Spring Preprints 2006. In Audio Engineering Society ‑ 120th Convention Spring Preprints 2006 (pp. 1–7).
- Simionato, R., Fasciani, S., and Holm, S. (2024). Physics‑informed differentiable method for piano modeling. Frontiers in Signal Processing, 3, 2023.
- Sinclair, S. (2009). Audio‑haptic interaction with the digital waveguide bowed string. In Proceedings of the International Computer Music Conference (ICMC 2009), Montreal, Canada.
- Smith, J. O. (1992). Physical modeling using digital waveguides. Computer Music Journal, 16(4), 74–91.
- Smith, J. O. (2002).
Principles of digital waveguide models of musical instruments . In Applications of Digital Signal Processing to Audio and Acoustics (pp. 417–466). Springer US. - Smith, J. O. (2004). Virtual acoustic musical instruments: Review and update. Journal of New Music Research, 33(3), 283–304.
- Smith, J. O. (2011). Spectral Audio Signal Processing. W3K.
- Tricco, A. C., Lillie, E., Zarin, W., O’Brien, K. K., Colquhoun, H., Levac, D., Moher, D., Peters, M. D. J., Horsley, T., Weeks, L., Hempel, S., Akl, E. A., Chang, C., McGowan, J., Stewart, L., Hartling, L., Aldcroft, A., Wilson, M. G., Garritty, C., … Straus, S. E. (2018). PRISMA extension for scoping reviews (PRISMA‑ScR): Checklist and explanation. Annals of Internal Medicine, 169(7), 467–473.
- Tuominen, P. P. A., and Saarni, L. A. (2024). The use of virtual technologies with music in rehabilitation: A scoping systematic review. Frontiers Virtual Reality, 5.
- Turchet, L., Hamilton, R., and Çamci, A. (2021). Music in extended realities. IEEE Access: Practical Innovations, Open Solutions, 9, 15810–15832.
- Uncini, A. (2022). Digital audio processing fundamentals (Vol. 21). Springer.
- Välimäki, V., and Erkut, C. (2003). Commuted waveguide synthesis of the clavichord. Computer Music Journal, 27(1), 71–82.
- Välimäki, V., Penttinen, H., Knif, J., Laurson, M., and Erkut, C. (2004). Sound synthesis of the harpsichord using a computationally efficient physical model. EURASIP Journal on Advances in Signal Processing, 2004(7), 860718.
- van den Oord, A., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A. W., and Kavukcuoglu, K. (2016).
WaveNet: A generative model for raw audio . In The 9th ISCA Speech Synthesis Workshop, SSW 2016 (pp. 125). ISCA. - Villa‑Renteria, I., Wang, M. L., Shah, Z., Li, Z., Kim, S., Ramachandran, N., and Pilanci, M. (2025). Subtractive training for music stem insertion using latent diffusion models. In 2025 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2025 (pp. 1–5). IEEE.
- Villeneuve, J., and Leonard, J. (2019). Mass‑interaction physical models for sound and multi‑sensory creation: Starting anew. In SMC 2019 ‑ 16th Sound & Music Computing Conference, Malaga, Spain.
- Wessel, D., and Wright, M. (2002). Problems and prospects for intimate musical control of computers. Computer Music Journal, 26(3), 11–22.
- Willemsen, S., Andersson, N., Serafin, S., and Bilbao, S. (2019). Real‑time control of large‑scale modular physical models using the Sensel Morph. In Proceedings of the 16th Sound and Music Computing Conference, Malaga, Spain: Sound and Music Computing Network.
- Willemsen, S., Paisa, R., and Serafin, S. (2020). Resurrecting the tromba marina: A bowed virtual reality instrument using haptic feedback and accurate physical modelling. In Proceedings of the 17th Sound and Music Computing Conference (pp. 300–307). Torino, Italy: Axea sas/SMC Network.
- Willemsen, S., Serafin, S., Bilbao, S., and Ducceschi, M. (2020). Real‑time implementation of a physical model of the tromba marina. In Proceedings of the 17th Sound and Music Computing Conference, Proceedings of the Sound and Music Computing Conference (pp. 161–168). Torino, Italy: Axea sas/SMC Network.
- Woodhouse, J., Politzer, D., and Mansour, H. (2021). Acoustics of the banjo: Measurements and sound synthesis. Acta Acustica, 5, 15.
- Worrall, K., Yin, Z., and Collins, T. (2024). Comparative evaluation in the wild: Systems for the expressive rendering of music. IEEE Transactions on Artificial Intelligence, 5(10), 5290–5303.
- Wu, Y., Manilow, E., Deng, Y., Swavely, R., Kastner, K., Cooijmans, T., Courville, A. C., Huang, C.‑Z. A., and Engel, J. H. (2022). MIDI‑DDSP: Detailed control of musical performance via hierarchical modeling. In The Tenth International Conference on Learning Representations, ICLR 2022.
OpenReview.net . - Wun, C.‑W., Horner, A., and Ayers, L. (2001). Perceptual wavetable matching for synthesis of musical instrument tones. In Proceedings of the 2001 International Computer Music Conference, ICMC 2001. Michigan Publishing.
- Yang, H. C., Lin, Y., and Su, A. (2020).
A novel source filter model using LSTM/K‑means machine learning methods for the synthesis of bowed‑string musical instruments . In 148th Audio Engineering Society International Convention 2020. Virtual Vienna. Audio Engineering Society. - Ye, Z., Xue, W., Tan, X., Liu, Q., and Guo, Y. (2023). NAS‑FM: Neural architecture search for tunable and interpretable sound synthesis based on frequency modulation. In Proceedings of the Thirty‑Second International Joint Conference on Artificial Intelligence, IJCAI 2023. (pp. 5869–5877).
ijcai.org . - Zhang, Z., and Akama, T. (2024). Hyperganstrument: Instrument sound synthesis and editing with pitch‑ invariant hypernetworks. In IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2024 (pp. 6640–6644). IEEE.
DOI: https://doi.org/10.5334/tismir.267 | Journal eISSN: 2514-3298
Language: English
Submitted on: Apr 8, 2025
Accepted on: Feb 24, 2026
Published on: Apr 13, 2026
Published by: Ubiquity Press
In partnership with: Paradigm Publishing Services
Publication frequency: 1 issue per year
Keywords:
© 2026 Yaozhong Zhang, Sebastian von Mammen, Christof Weiß, published by Ubiquity Press
This work is licensed under the Creative Commons Attribution 4.0 License.