Figure 1
The return to Tempo I during the second movement of Rachmaninoff’s second piano concerto requires a highly coordinated musical effort. The flutes (‘Fl.,’ top line of second system) sustain a fermata, while the piano soloist interprets a closing ornamentation at their own tempo. All parties are expected to land in complete synchronization on the following cadential downbeat, most often facilitated by a conductor actively listening to, observing, and communicating with the three musicians.
Figure 2
This illustration depicts our framing of the flow of information within the human–machine performance system, with arrows added to highlight ideas we build into the framework. The flow from system output to machine intention is the traditional feedback loop that gives the machine the ability to intelligently construct new intentions based on previous output. The co‑creative line shows this creative input, by which the output produced by the machine influences the creative intentions of the human agent.
Figure 3
The system looks for two gestures, ‘hand raised’ and ‘hand to head,’ which indicate a musician’s desire to move to a major or minor chord, respectively.
Figure 4
(Top) The ImproVision robot pans to look at a particular musician. It will then tilt up or down, corresponding to the desired note change for this musician. (Bottom) To indicate an ensemble down‑beat, the robot pans to the center, tilts up, and then tilts down.