A Lightweight Two‑Branch Architecture for Multi‑Instrument Transcription via Note‑Level Contrastive Clustering

Datasets used in experiments. ‘Our:AMT’ and ‘Our:Sep’ are our synthetic datasets created to examine whether human composition and real recordings are indispensable. The latter’s timbres are grouped into 10 classes; when training, cross‑class mixtures yield single‑sample multi‑timbre pieces. Bold datasets are used for training.

Comparison of timbre‑agnostic transcription. Unless stated otherwise, models are trained on our synthetic dataset. ‘noLog’: EnergyNorm without log; ‘BN’: BatchNorm replacing EnergyNorm (BasicPitch‑style); ‘Conv39’: BasicPitch’s 39‑tap conv replacing our dilated conv; ‘BPloss’: BasicPitch’s loss; ‘MN’: trained on MusicNet; ‘lCQT’: learnable CQT; ‘l${\mathbf{\text{CQT}}}_{\mathbf{\text{MN}}}$’: learnable CQT trained on MusicNet; ‘BP’: re‑implementation of BasicPitch (Bittner et al., 2022); ‘${\mathbf{\text{BP}}}^{\mathbf{\text{fl}}}$’: BP with focal loss; ‘${\mathbf{\text{BP}}}_{\mathbf{\text{MN}}}^{\mathbf{\text{fl}}}$’: ${\text{BP}}^{\text{fl}}$ trained on MusicNet; ‘OF’: re‑implementation of Onsets&Frames (Hawthorne et al., 2018) with its original BCE loss; ‘${\mathbf{\text{OF}}}^{\mathbf{\text{fl}}}$’: OF with focal loss; and ‘${\mathbf{\text{OF}}}_{\mathbf{\text{MN}}}^{\mathbf{\text{fl}}}$’: ${\text{OF}}^{\text{fl}}$ trained on MusicNet. In headers, ‘tutti’ = full‑mix polyphonic pieces; ‘stems’ = single‑instrument samples.

Comparison of timbre‑separated transcription. Unless stated otherwise, models are trained on MusicNet with InfoNCE loss (8). ‘D16’: $D=16$; ‘MSE’: using $L_{\text{affinity}}$(6); ‘Syn’: trained on our synthetic dataset; ‘Rescale’: forcibly scaling amplitude using Frame prediction before InstanceNorm in the timbre‑encoding branch; ‘Share’: sharing the first residual block between two branches; and ‘Tanaka’: baseline model (Tanaka et al., 2020) using InfoNCE. All experiments use identical pretrained AMT branch parameters except ‘Share.’