Abstract
In this work we implement an interactive system for filling in missing measures in a monophonic music piece. This system takes a user’s hand-drawn curves as input and generates a melody whose rhythm and pitch contour match with the curves. Contrary to previous interactive music inpainting work, users of the proposed system do not need to understand the music notation; they just need a rough idea of the shape of the melody and draw it out. This system is implemented under the variational auto-encoder framework and is enabled by a proposed melody disentanglement scheme to disentangle relative pitch, relative rhythm and musical context. We also create a web-based graphical user interface to facilitate the user interaction. We evaluate the system on a commonly used Irish folk song dataset. Objective and subjective evaluations show that this novel interaction is intuitive and effective for melody inpainting, and the proposed neural approach outperforms two baselines we developed based on previous work, in terms of musicality and fidelity to the user’s input.