ChoraleBricks: A Modular Multitrack Dataset for Wind Music Research

2.1 Wind music research

The historical development of brass and woodwind music in Europe began with ancient military signals and evolved through the Renaissance into organized bands for civic and ceremonial functions (Hofer, 1992). In the 18th and 19th centuries, professional military bands and the popular brass band movement emerged. This genre has evolved from its origins to develop its own distinct tradition, reflecting Europe's rich musical heritage. In particular, brass and woodwind music is an integral part of contemporary German culture, with military, church‑related, and purely civilian concert bands and ensembles performing frequently. With a focus on the history and relevance of American wind music, the development from military music to modern wind music in the 20th century is described by Battisti (2018). A significant milestone in this development was the formation of the Eastman Wind Ensemble (EWE) by Frederick Fennell in 1952. The EWE emphasized flexible instrumentation, allowing for a wide repertoire ranging from classical transcriptions to contemporary works. In Chapter 24, Battisti (2018) states: “All new technological devices and instruments should be used to promote wind bands/ensembles and their music. Every iPhone and iPad is a concert hall.”

From a scientific perspective, wind music is studied by small yet active research communities around the world, focusing mainly on musicological aspects. Two examples of such communities are the Internationale Gesellschaft zur Erforschung und Förderung der Blasmusik (IGEB)⁵ and the World Association for Symphonic Bands and Ensembles (WASBE).⁶ At universities, research on concert band and wind music is primarily conducted at musicological institutes or within performance studies. A notable research institute is the Pannonische Forschungsstelle/Internationales Zentrum für Blasmusikforschung,⁷ which focuses on wind music research and houses one of the most prolific libraries dedicated to wind music in the world. Additionally, it serves as the headquarters of the aforementioned IGEB.

2.2 Multitrack datasets

In the past, some multitrack datasets have been released for research purposes, e.g., MUSDB18 (Rafii et al., 2017), MedleyDB (Bittner et al., 2014), or MoisesDB (Pereira et al., 2023). However, most of these datasets concentrate on popular and rock music, where brass instruments and woodwinds usually play a minor role. An exception is the Cadenza Woodwind dataset, which was released during Cadenza's Second Open Machine Learning Challenge (CAD2) for the task of rebalancing classical music ensembles (Roa Dabike et al., 2024).⁸ In this case, the audio recordings in the dataset stem from a synthetic data generation pipeline, an emerging trend which could be observed in other datasets as well, such as in EnsembleSet (Sarkar et al., 2022) or the CocoChorales dataset (Wu et al., 2022). Further symbolic encodings for chorales can be found in the Chorale Corpus (Gerhardt and Kirsch, 2024).⁹ Although high amounts of data can be generated, the versatility in timbre, articulation, and modulation of some instruments (e.g., trumpets) is not yet comparable to real recordings.

In the following, we discuss the most influential works for this paper and the ways in which the ChoraleBricks dataset is continuing the existing line of research. On a conceptual level, our work on ChoraleBricks has been inspired by three influential datasets. First, Duan and Pardo (2011) created the dataset Bach10 by recording ten Bach chorales using violin, clarinet, saxophone, and bassoon. Each part was recorded sequentially, starting with the violin, while successive players listened to the previously recorded parts to align their tempo and dynamics. Second, the Operation Beethoven dataset by Kaiser et al. (2023), involved an orchestra recording a reference performance of the first movement of Beethoven's 4th Symphony (approximately 10 minutes). Each instrumental section then recorded their parts individually using the reference track as a guide, although the recordings were not strictly isolated. Finally, University of Rochester Multi‑Modal Music Performance (URMP) by Li et al. (2019) features 44 pieces with various instruments, totaling 1.2 hours of isolated multitrack recordings. The methodology included a prerecorded video of a conductor accompanied by a pianist. Subsequent musicians were provided with this video and the piano audio as a guide track to ensure alignment during their recordings.

These datasets significantly influenced ChoraleBricks' design and methodology. While our dataset builds upon Bach10's approach, we integrate methodological insights from URMP, such as alignment strategies, to enhance the recording process. The key contribution of ChoraleBricks lies in its diversity of instruments, enabling dynamic creation of new mixes with varying complexity, thus broadening its utility for downstream tasks.

3.1 Work selection

The first step involves selecting the works to include in the dataset. In ChoraleBricks, each work is a four‑part chorale consisting of soprano (S), alto (A), tenor (T), and bass (B) parts, with the soprano voice presenting the melody. During the work selection process, we focused on three key aspects: First, the chosen works should be representative of the chorale literature, particularly within the German tradition. Second, the arrangements should exhibit a homogeneous style and a comparable level of complexity—short and accessible enough for amateur musicians to perform. Finally, and crucially, the arrangements must be eligible for publication under an Open Access license.

A common resource in chorale music literature is the chorale accompaniment book, which typically provides four‑part chorales corresponding to the entries in a related hymn book. One such accompaniment book is the Neues Thüringer Choralbuch (English: New Thuringian Chorale Book), originally compiled by Rudolf and Erhard Mauersberger (Mauersberger and Mauersberger, 1955).¹⁰ Although initially designed for organists, some of the chorale arrangements included in the book are derived from choir literature. The movements adhere to a strict arrangement with four monophonic parts. The counterpoint is predominantly homophonic, meaning all parts follow the rhythm of the melody, which (referred to as cantus firmus in Latin) is always in the soprano. The pitch ranges are comfortably singable by an SATB choir and playable by brass ensembles such as a Posaunenchor or woodwind groups. While the Neues Thüringer Choralbuch was initially created to accompany two now‑outdated hymn books (Deutsches Evangelisches Gesangbuch and Evangelisches Kirchengesangbuch), its chorale arrangements remain relevant. The melodies frequently appear in modern hymn books, and the four‑part chorales continue to represent today's practices, offering flexible instrumentation options for a variety of ensembles.

Given the stylistic homogeneity of the arrangements in this particular chorale book (Baroque style with predominantly homophonic counterpoint), along with their relatively low technical skill requirements and flexibility for various instrumentations, we chose this book as the foundation for our dataset. For ChoraleBricks, we selected ten works from this book as a representative subset, as shown in Table 1. The selected works date back to the 16th, 17th, and 18th centuries and include compositions by renowned composers such as Bach, Telemann, and Vulpius, among others.

Figure 3

Figure 3 presents a score rendition of a typical chorale from the dataset (“Auf, auf, mein Herz, mit Freuden” by Johann Crüger (CR1)). In the score, the first and second voices (soprano and alto) are notated in the treble clef on the first and second staff lines, while the third and fourth voices (tenor and bass) are notated in the bass clef.

In ChoraleBricks, the musical scores are provided in multiple formats. First, we use the MusicXML format. As shown in Figure 3, each voice is placed on a separate staff line, and all repetitions are expanded by duplicating the respective measures. These MusicXML encodings form the foundation for symbolic music representations utilized in the audio‑to‑score alignment processes discussed later in the paper.

Additionally, the scores are encoded in the MEI symbolic music format (version 5.0), established as the international standard for digital scholarly music editions (Hankinson et al., 2011; Rol, 2023). MEI's capabilities for annotating critical editions, analytical data, and variances across different versions of the Neues Thüringer Choralbuch make it ideal for this project and potential future expansions.

The graphical scores, such as the one shown in Figure 3, were rendered using MuseScore v4.0+ (Muse Group, 2022). In the original sheet music, the soprano and alto parts were combined as two voices in a single staff line in the treble clef, while the tenor and bass parts were combined in the bass clef. Although most musicians are accustomed to reading from the full score during recording sessions, the additional MusicXML encoding, derived from MEI, provides a flexible and standardized format for computational tasks.

3.2 Recording of the conducting video

A key insight from the URMP dataset preparation was that conducting videos effectively ensured synchronization across sequentially recorded tracks (Li et al., 2019). Building on this approach, we created conducting videos for all ten chorales. The videos feature a clear and accessible conducting style (performed by the first author) and tempi consistent with common performance practices. Each video begins with an inviting gesture and a smile toward the musician, typically followed by one measure with preparatory beats to establish the tempo. The chorales conclude with a fermata to signal the ending.

Figure 4a illustrates the technical setup for the video recordings. Various backgrounds, including white walls and sheets, were tested before settling on a gray paper backdrop commonly used in photography. Lighting was provided by a single light source positioned behind a diffuser on the right side, complemented by a reflector on the left to soften shadows. The recordings were made using a standard smartphone (Apple iPhone 13) mounted on a tripod approximately 2 meters in front of the conductor. Figure 4b presents a screenshot from one of the conducting videos as an example. It is worth noting that the lighting and background setup evolved during the video recordings, resulting in minor inconsistencies in appearance across the videos.

Figure 4

Although not entirely consistent, we decided to include the conducting videos in the dataset for two key reasons. First, they document the recording process and may offer a research avenue, such as analyzing the relationship between conducting tempo and performed tempo. Second, the videos could prove highly valuable for expanding the dataset in the future, enabling other musicians and researchers to contribute, potentially through web‑based experiments.

3.3 Recording of the individual tracks

The third step, depicted in Figure 2, is central to the dataset creation process: recording the individual tracks. As noted earlier, the primary objective of the ChoraleBricks dataset is to provide completely isolated recordings of each voice. To achieve this, musicians were recorded sequentially—a method that, while effective for ensuring data modularity, deviates significantly from traditional musical practices and the way this repertoire is typically performed. Recognizing this limitation, we carefully navigated the trade‑off between developing an analytically robust and highly modular dataset and maintaining the natural musicality of the performances.

As for ChoraleBricks, we prioritized the flexibility afforded by a controlled recording process and setup. Figures 4c and 4d illustrate the arrangement used during the recording sessions. The conducting video was played back via a digital audio workstation (Apple Logic Pro, v11.1.1) and displayed on an iPad as a secondary screen, ensuring synchronization between the video and audio during the recording process.

For tuning, we aimed for A₄ = 442 Hz, a standard frequency commonly used in modern wind ensembles and concert bands. Recordings were captured with a high‑quality condenser cardioid microphone (Schoeps MK 4), positioned approximately 2 meters from the musician. This setup aimed to capture the sound as perceived by an audience or conductor rather than the dry sound typically produced by close‑up or clip microphones. The microphone's angle was adjusted on the basis of the instrument, following recommendations from experience and relevant literature (Albrecht, 2017).

The sessions were conducted in three different environments: a rehearsal room (in Bödexen), a recording studio (in Erlangen), and an office (in Höxter). All locations featured acoustically controlled conditions, free from the excessive reverberation often found in churches or similar spaces.

In most chorales, the soprano and tenor voices were typically recorded first, often performed by the trumpet or flügelhorn. In some cases, however, the bass voice was recorded first. This was the case when a baritone horn player recorded the initial tracks of a chorale. Due to its pitch range, it is possible to record all four voices with the baritone horn (e.g., in VU2). Both strategies provided a reference for subsequent musicians, who could listen to the previously recorded tracks while following the conductor's movements during their own performance. This approach aligns with observations from the Bach10 dataset, where the initial player effectively “sets the scene” (Duan and Pardo, 2011). Similar to the methodology used in URMP (Li et al., 2019), the inclusion of a conducting video in our setup served as a valuable visual cue, ensuring consistent timing across all recordings.

Musicians reported that, as the number of prerecorded tracks increased, it became easier to play along with the audio. Once enough tracks were available, they could request specific mixes or receive tailored ones on the basis of their preferences. A common preference was for mixes featuring similar instruments. For example, saxophone players found it easier to match intonation with other reed instruments, while brass players favored mixes dominated by brass instruments. Although not systematically documented, these preferences align with intuitive musician behavior: Familiar instruments provide more comfortable references for both recording sessions and live performances.

3.4 Data curation

In the fourth step, as illustrated in Figure 2, the data were curated and enriched with additional annotations, including score‑based audio annotations. This section details the post‑processing steps, annotation procedures, and alignment processes, ensuring high‑quality data preparation and comprehensive metadata to support diverse applications in MIR (see also Section 5).

3.4.2 Annotations

For annotations, we provide manually corrected fundamental frequency (F₀) trajectories and aligned note events (similar to a piano‑roll representation) for each track.

F₀ Annotations

The F₀ trajectories were annotated using Sonic Visualiser (v5.0.1) and the pYIN VAMP plugin (v3) (Cannam et al., 2010; Mauch and Dixon, 2014), except for the tuba, where a different procedure was required owing to its very low pitches (details provided below). Figure 5a illustrates an example of the resulting annotations for a flute track. On the last note, regular amplitude modulations in the waveform and frequency modulation in the extracted F₀ trajectory (red curve) can be observed, producing a characteristic vibrato typical of many flute recordings. The F₀ trajectory was derived using the “pYIN: Smoothed Pitch Track” function with standard settings.

Figure 5

Additionally, a note track (Figure 5a, gray boxes), similar to a piano‑roll representation, was extracted using the “pYIN: Notes” function with the same settings and adjustments as for the F₀ trajectories. Built‑in sonification was used to verify both the F₀ trajectory and the note tracks. Manual corrections were necessary for the note tracks, particularly when consecutive notes were not accurately identified or when note onsets were audibly delayed.¹³ In cases of F₀ trajectory errors (primarily octave errors), a melodic range spectrogram was employed to manually correct the trajectory on the basis of the spectrogram data. For the tuba recordings, where low‑note extraction quality was unsatisfactory, a different method was used. Note events were first extracted using Sonic Visualiser. Then, a salience‑based F₀ estimator was adapted to incorporate constraint regions defined by the note events (Müller and Zalkow, 2021).¹⁴ Finally, both the F₀ trajectories and note tracks were exported as CSV files for each track.

The high quality of these manually corrected and verified annotations makes them well suited for tasks such as assessing pitch estimation methods in terms of pitch accuracy. The sonification process proved instrumental in detecting and rectifying voicing errors. Additionally, the pYIN implementation operated at a resolution of 20 cents, followed by parabolic interpolation, surpassing the 50‑cent resolution commonly employed by evaluation toolkits such as mir_eval (Raffel et al. 2014). However, two potential confounding factors should be considered when using them as a reference for evaluation. First, initial annotations were generated using the probalistic YIN (pYIN) algorithm, which may introduce a bias toward this approach. Second, determining note onsets and durations is inherently ambiguous, and the annotations were adjusted by ear when necessary. This process could introduce uncertainty in note boundaries, potentially affecting evaluation metrics focused on the voicing capabilities of pitch estimators.

Aligned score annotations

In audio‑to‑score alignment, the task is to link recorded performances in physical time (given in seconds) to sheet music in musical time (given in measures), as described by Müller (2015). This process is often labor‑intensive when performed manually. ChoraleBricks simplifies it by leveraging F₀‑annotated note events and monophonic note sequences derived from the sheet music. This enables a direct mapping between the F₀ annotations and note events extracted from MusicXML files using the music21 library (Cuthbert and Ariza, 2010).

Figure 5 illustrates the annotation process: The note events from the F₀ annotations (Figure 5a, gray boxes) served as the basis for the performed notes in the audio recordings. Frequency values were converted into MIDI note numbers. On the sheet music side (Figure 5b), the corresponding note events were extracted from the “unrolled” MusicXML encoding (Figure 3) using the Python library music21. Once the number of note events was confirmed to match between the audio and sheet music, both sequences were linked (Figure 5, red arrows). Finally, pitch alignment between the audio and score was verified through sonifications using libsoni (Özer et al., 2024).

The aligned scores were exported to a CSV file, with each line representing a note event specified by its physical time in the audio and its measure position, encoded as a real number with a fixed precision of three digits. The integer component indicates the measure, while the fractional component represents the relative position within the measure. For instance, the note events from Figure 5b correspond to (start:end): (0.833:1.000), (1.000:1.333), (1.333:1.500), (1.500:1.833), (1.833:2.000), and (2.000:2.833).

This approach, however, does not retain connections to the original MusicXML note objects. To address this, we provide a second export using the Music Performance Markup (MPM) format (Berndt, 2022), an extension to symbolic music formats such as MEI and MusicXML with additional performance properties. Combined with the MPM Toolbox¹⁵ that provides built‑in functionalities for alignment to audio recordings, these exports enable the creation of synthesized renditions with fine‑grained control over performance parameters such as timing, dynamics, and articulation.

Each track in the ChoraleBricks collection includes a corresponding alignment file containing sheet music information and its link to the audio. The precise mapping between F₀‑annotated note events and the corresponding MusicXML‑derived note sequences guarantees consistency across the dataset. The annotations were subjected to thorough manual verification, including sonification for detecting alignment errors. Verification of the alignments is subject to future investigations; however, we assume that alignment deviations stay below 50 milliseconds. While the alignments share the same limitations regarding note onset and duration ambiguities as the F₀ annotations do, their integration with ChoraleBricks' modular mixing capabilities makes the dataset well suited for a variety of systematic experiments.

6.1 F₀ Estimation and transcription

F₀ estimation and automatic music transcription (AMT) are foundational areas in MIR, with state‑of‑the‑art methods often relying on neural‑network‑based approaches (Kim et al., 2018). However, applications such as real‑time intonation monitoring require low‑latency estimates, where traditional methods such as YIN (de Cheveigné and Kawahara, 2002) or the sawtooth waveform inspired pitch estimator (SWIPE) (Camacho and Harris, 2008) remain viable owing to their simplicity and efficiency. In such contexts, robustness and efficiency are just as important as accuracy.

ChoraleBricks, with its clean, isolated recordings, provides a robust framework for testing F₀ estimators under diverse conditions, such as simulated crosstalk in ensemble performances. Additionally, the dataset can mitigate biases inherent in annotations (e.g., pYIN‑based errors or subjective onset/offset discrepancies) through synthetic approaches using sinusoidal models (Salamon et al., 2017), enabling more objective evaluations across different F₀ estimation methods.

In AMT, early research primarily targeted piano recordings, driven by large datasets such as MIDI and Audio Edited for Synchronous Tracks and Organization (MAESTRO) (Hawthorne et al., 2019). Recent advancements, including transformer‑based models (Maman and Bermano, 2022), have broadened transcription applications to other instruments. However, their performance on diverse datasets such as URMP (Li et al., 2019) remains limited. ChoraleBricks, with its varied instrumentation and modular design, offers an essential resource for evaluating and enhancing AMT models, addressing transcription challenges across a broader spectrum of instruments.

6.2 Intonation analysis

In ensemble performances, achieving rhythmic precision and balanced intonation is crucial for a cohesive sound. While ensembles tune their instruments to a reference pitch (e.g., A₄ = 442 Hz) at the start of a session, factors such as temperature and humidity, particularly for woodwind instruments, can lead to intonation shifts during rehearsals or concerts. Players adjust their tuning dynamically through subtle variations in their playing techniques rather than altering the instrument itself.

Harmonic context also necessitates pitch adjustments, as notes in a major triad (e.g., root, third, or fifth) often require fine‑tuning to align with harmonic expectations. Studies such as that of Howard (2007) have shown intonation drifts in vocal ensembles, often due to modulations, with singers tending to non‑equal temperament tunings. Similar analyses could be conducted for wind ensembles using the ChoraleBricks dataset.

ChoraleBricks' modular design enables the exploration of various instrument combinations and harmonic complexities, offering opportunities to define or learn cost measures for intonation quality (Schwär et al., 2021). Such measures could help identify well‑intonated ensembles and investigate phenomena, such as whether larger ensembles achieve better intonation than smaller ones—an observation often noted in amateur settings.

6.3 Audio alignment

Audio–score alignment, along with related problems such as score following and computer accompaniment, has been extensively studied (Dannenberg and Raphael, 2006; Müller et al., 2019). While much of this research focuses on piano music, orchestral and opera music pose unique challenges owing to their complexity. In particular, inconsistencies arising from musical and performance‑specific factors further complicate the alignment of score elements, such as notes or measures, with corresponding audio positions. These asynchronies between orchestral and vocal elements are commonly found in opera recordings (Weiß et al., 2016).

In this context, the ChoraleBricks dataset offers valuable opportunities for studying these challenges by providing alignments at the track level, allowing for evaluations at the instrument level. This level of detail enables researchers to explore alignment behaviors more thoroughly, such as identifying which instruments an alignment approach tends to prioritize, for example, trumpets in the soprano voice or tuba in the bass voice. Such analyses yield important insights into the performance and refinement of alignment methods, contributing to advancements in audio‑to‑score alignment research.

6.4 Music education

Music education apps such as Yousician¹⁶ and flowkey¹⁷ have become popular tools for beginners, offering features such as just‑in‑time feedback while practicing. Most apps focus on widely used instruments such as piano, guitar, and bass. Some, such as Songs2See,¹⁸ also support wind instruments. For more advanced aspects, such as intonation and sound quality, specialized tools such as Korg Cortosia analyze tone properties, including pitch and timbre stability (Bandiera et al., 2016).

The ChoraleBricks dataset could expand these educational approaches by facilitating interactive training environments for ensemble playing, potentially through a web interface. For example, students could practice specific voices from chorales and receive feedback on accuracy and timing. As proficiency increases, they could transition to play‑along scenarios with customizable challenges, such as changing instruments, adding reverberation effects, or simulating tuning shifts. Additionally, user‑generated recordings could extend the dataset with crowdsourced performances, creating a valuable resource for evaluating F₀ estimation and transcription methods.

6.5 Music source separation

Music source separation (MSS) seeks to isolate and extract individual audio components (e.g., vocals, drums, bass, guitar) from a mixed recording, allowing the creation of separate tracks for each source. Recent progress in MSS has been driven by efforts such as the Sound Demixing Challenge (Fabbro et al., 2024) and datasets such as MUSDB18 (Rafii et al., 2017) and MoisesDB (Pereira et al., 2023). While MSS systems excel with Rock and Pop instruments, current research aims to achieve universal source separation to address instrument‑specific limitations (Watcharasupat and Lerch, 2024). With its diverse instrumentation and isolated tracks, the ChoraleBricks dataset provides a valuable resource for advancing and evaluating MSS techniques.

6.6 Music generation

Recent advancements in generative models, such as denoising diffusion probabilistic models (DDPM) (Hawthorne et al., 2022; Maman et al., 2024), leverage large datasets to synthesize audio directly from symbolic music representations. To reduce data requirements while enhancing interpretability, model‑based deep learning combines traditional knowledge‑driven methods with data‑driven techniques within a differentiable computing framework (Richard et al., 2024). An example is the differentiable digital signal processing (DDSP) framework (Engel et al., 2020), which integrates digital signal processing (DSP) elements into deep learning pipelines and has gained prominence in musical sound synthesis.

ChoraleBricks contributes to advancing music generation, particularly for wind instruments, by offering high‑quality, modular recordings and precise per‑track note‑wise annotations. These resources support the development of generative models capable of synthesizing realistic performances from symbolic music data. For instance, ChoraleBricks may facilitate research using DDPMs, DDSP, and hybrid frameworks, helping to address confounding factors and enabling innovative ensemble generation with detailed control over timbre, articulation, and dynamics.