Cross-Cultural Music Similarity: Bridging Human Perception, Signal Processing, and Foundation Models

2.1 Foundation models for music and audio

The emergence of large‑scale pre‑trained models has significantly advanced audio and music understanding capabilities (Ma et al., 2024). Early work in this direction adapted vision and language model architectures to audio domains, with models like Wav2Vec (Baevski et al., 2020) and Audio‑MAE (Huang et al., 2022) demonstrating the effectiveness of self‑supervised learning on audio representations.

More recently, specialized foundation models for music have emerged. MERT (‘Music Understanding Model with Large‑Scale Self‑supervised Training’) (Li et al., 2024) uses a BERT‑style architecture (Devlin et al., 2019) with 12 Transformer encoder layers and approximately 95 million parameters (with a larger 330M variant with 24 layers also available). Pre‑training involves the employment of masked acoustic modeling, using an acoustic and a musical teacher. The model demonstrates strong performance across various downstream tasks, including music classification, tagging, and similarity assessment.

CLAP (‘Contrastive Language‑Audio Pre‑training’) (Wu et al., 2023) extends contrastive learning principles from CLIP (Radford et al., 2021) to the audio domain, learning joint representations of audio and text. The CLAP‑Music variant is trained specifically on musical content, while CLAP‑Music&Speech extends the training data to include both music and speech content, creating a more generalized audio‑language model.

Other notable foundation models include Qwen‑Audio (Chu et al., 2023), which represents a unified audio understanding foundation model capable of processing both speech and music. Its Qwen2‑Audio variant (Chu et al., 2024) is pre‑trained on a diverse collection of audio data using multiple objectives, which potentially enables better transfer between different audio domains, including across musical traditions (Papaioannou et al., 2025).

2.2 Cross‑Cultural music studies in MIR

Cross‑cultural music analysis has gained increasing attention in the MIR community, driven by recognition of the field’s historical Western bias and the need for more inclusive approaches (Mehta et al., 2025; Serra et al., 2013). Prior work (Panteli, 2018) highlighted fundamental challenges in applying Western‑centric features to non‑Western musical traditions, particularly regarding microtonal systems and non‑equal temperament scales. The assumption of music as a ‘universal language’ is challenged (Mehr et al., 2019; Savage et al., 2015), with the cultural context influencing auditory perception and aesthetic appraisal, leading to diverse ‘listening frameworks’ and ‘musical ontologies’ (García‑Benito, 2025).

Several studies have explored culture‑specific adaptations of MIR techniques to address fundamental methodological limitations in analyzing diverse musical systems. These limitations manifest at multiple levels, from low‑level feature extraction to high‑level task formulation (Gómez et al., 2013). For instance, chroma features implicitly assume 12‑tone equal temperament, making them less suited for traditions with different tuning systems (Müller et al., 2018), while rhythmic features often assume certain metrical hierarchies, struggling with complex cycles of traditions like Carnatic music (Srinivasamurthy et al., 2014a).

Research efforts have pursued two primary approaches to address these challenges. One direction involves developing specialized representations for specific traditions, such as MIDI‑based representations for Turkish makam (Karaosmanoğlu et al., 2014), specialized models for makam music lyrics‑to‑audio alignment (Dzhambazov et al., 2016), and methods for identifying asymmetric rhythms in Greek music (Fouloulis et al., 2013). While effective for specific traditions, these specialized approaches lack scalability across multiple musical systems.

The alternative direction employs end‑to‑end learning approaches that minimize inductive bias, allowing models to discover relevant patterns without imposing predefined notions of musical importance (Serra, 2014). However, learned representations from deep models inherit biases from their training data. Recent work has highlighted cultural biases in music‑generation models like Mustango and MusicGen, stemming from Western‑centric training corpora (Mehta et al., 2025). While the specific training datasets for these generation models may differ from those used for the audio foundation models we evaluate, the broader challenge of Western‑dominated training distributions applies across audio model development, creating a ‘self‑reinforcing cycle’ where established datasets lead to specialized algorithms, which in turn encourage more similar data collection (Sturm, 2013). This motivates our systematic cross‑cultural evaluation approach.

Recent work has begun to address bias and fairness in cross‑cultural music analysis (Holzapfel et al., 2018; Kanatas et al., 2025; Papaioannou et al., 2025), highlighting the need for more representative datasets and evaluation frameworks that account for diverse musical perspectives.

2.3 Human perception and music similarity

Music similarity in MIR research has been conceptualized through diverse paradigms including commonality‑based methods (Meredith, 2014), transformational approaches (Lewin, 1982), learned adaptive mappings (Bello, 2011), and context‑dependent measures acknowledging that listeners dynamically adjust judgments based on task and cultural background (Krumhansl and Castellano, 1983). Our study focuses on distance‑ based similarity assessment, representing one important but not exhaustive approach to computational similarity evaluation.

Understanding how humans perceive musical similarity through this lens has been a central question in music cognition and MIR research. Early work (Eerola et al., 2001) investigated statistical features and perceived similarity of folk melodies, finding that frequency‑based musical properties could account for moderate amounts of listeners’ similarity ratings, with descriptive variables like melodic predictability and rhythmic variability achieving slightly better performance. Research examining similarity perception across musical styles (Lamont and Dibben, 2001) found that judgments were context‑specific and roughly equivalent between trained musicians and non‑musicians, with ratings primarily based on surface features such as dynamics, articulation, texture, and contour rather than deeper structural relationships.

Large‑scale evaluations have provided crucial insights into the relationship between computational approaches and human perception. A comprehensive cross‑site evaluation (Berenzweig et al., 2004) compared acoustic techniques against subjective measures across hundreds of popular artists, demonstrating that acoustic measures could achieve agreement with ground‑truth data comparable to internal agreement between different subjective sources. Recent work (Tian et al., 2025) evaluated audio representations against human timbre similarity ratings, finding that style embeddings from foundation models achieved superior performance compared to traditional signal processing features.

These studies demonstrate that human music similarity perception is multifaceted, involving both immediate surface features and deeper structural relationships, with significant individual and cultural variation. However, most existing research has focused on Western musical contexts or specific perceptual dimensions in isolation. Understanding similarity across cultural boundaries remains a significant challenge, requiring comprehensive evaluation approaches that can systematically compare human judgments against both traditional signal‑processing methods and modern foundation models while identifying the interpretable factors that drive cross‑cultural similarity perception.

4.1 Survey design and methodology

Pairwise Comparison Framework: Following established methodologies in music perception research (Hawthorne et al., 2018; Nawrot, 2003), we employed a pairwise comparison approach where participants evaluated randomly selected pairs of 20‑s audio clips. Each participant assessed 10 unique pairs, with pairs distributed to ensure comprehensive coverage across all dataset combinations while avoiding participant fatigue.

For each audio pair, participants provided ratings on three distinct similarity dimensions using a 9‑point Likert scale, with scores ranging from 1 to 5 points with steps of 0.5 points:

1. Overall Musical Similarity: ‘How similar are the two audio clips overall?’

2. Cultural Similarity: ‘How similar are the two audio clips in their cultural characteristics?’

3. Recommendation‑level Similarity: ‘How probable is it for you to put the two audio clips in the same playlist?’

We acknowledge that recommendation‑level similarity is inherently context‑dependent. Listeners may create playlists based on diverse criteria: personal taste, thematic coherence (e.g., 80s party theme), or situational appropriateness (e.g., relaxing evening). Our recommendation‑level similarity question asked participants to rate the general likelihood of including two tracks in the same playlist, capturing an average assessment across these potential contexts rather than any specific playlist type.

The three‑dimensional framework allows examination of how different aspects of similarity align or diverge, particularly important for cross‑cultural analysis where musical and cultural similarity may not coincide (Volk and Van Kranenburg, 2012).

Survey Implementation: The survey was implemented as a web‑based application ensuring cross‑platform compatibility and ease of access. Participants were recruited through academic networks, social media, and music communities, with an emphasis on achieving demographic diversity.

Ethics and Informed Consent: This study received ethics approval from the relevant institutional committee as minimal risk research (see the Ethics and Consent statement). All participants provided informed consent via an online form before accessing the survey and were informed of study procedures and their right to withdraw, and no personally identifiable information was collected.

4.2 Participant demographics and data statistics

Our study collected responses from 125 participants across 21 countries, resulting in 1,130 unique annotated pairs covering 463 unique audio clips. Table 2 summarizes the key study statistics. The participant pool demonstrated significant diversity across multiple dimensions: age distribution spanning 18–64 years with majority representation in the 25–44‑year range, varied musical training levels from no formal training (13.0%) to professional musicians (13.0%) and musicologists (11.7%), and familiarity with 58 different musical cultures.

Our participant pool includes individuals from 21 countries with the following geographic distribution: Europe 84.8% (Greece 62.4%, Italy 4.8%, France 4.0%, United Kingdom 4.0%, others 9.6%), Asia 8.8% (China 5.6%, others 3.2%), North America 3.2%, and other regions 3.2%. While Greek representation is substantial, the observed geographic and cultural diversity enhances cross‑cultural validity by providing multiple perspectives on music similarity across traditions. Nevertheless, we acknowledge that this distribution does not comprehensively represent all world musical cultures, and findings should be interpreted with this limitation in mind. Detailed demographic breakdowns, including age, gender and cultural familiarity distributions; musical training categories; and geographic origins are provided in the supplementary material.

5.1 Similarity computation for signal‑processing features

For each musical dimension, we computed similarity scores between audio pairs by combining multiple complementary features. The extraction methods and individual features are detailed in the supplementary material; here, we describe how these features are aggregated into dimensional similarity scores.

Melody Similarity ($S_{\mathbf{\text{melody}}}$): Combines pitch class distributions at two resolutions (quarter‑tone for microtonal systems, semitone for cultures using 12‑tone equal temperament), melodic interval patterns, and F0 statistical characteristics. The quarter‑tone resolution captures ornamentations and microtonal inflections present in many non‑Western traditions, while interval patterns reflect melodic motion independent of absolute pitch. We note that our melody analysis employs PYIN (Mauch and Dixon, 2014) for fundamental frequency estimation, which assumes predominantly monophonic melodic content. In polyphonic musical textures, common across many traditions, PYIN tracks the most salient F0 trajectory, which may not always correspond to the perceived melodic line. This means our melodic features capture prominent pitch content rather than precisely extracted melodies, potentially affecting reliability across different musical textures and arrangements.

Rhythm Similarity ($S_{\mathbf{\text{rhythm}}}$): Aggregates tempo, onset density and timing patterns, beat interval regularity, and tempogram profiles. This multi‑faceted approach captures both the perceived pulse (tempo and beat tracking) and the density of musical events (onset patterns), accommodating diverse rhythmic organizations from steady Western meters to complex polyrhythmic structures.

Harmony Similarity ($S_{\mathbf{\text{harmony}}}$): Integrates chroma features at dual resolutions, chord distributions and transitions, key profile correlations, and tonnetz tonal centroids. We acknowledge that our harmony features incorporate Western‑centric assumptions (major/minor chord templates, Krumhansl–Schmuckler key profiles) which may not adequately capture harmonic relationships in traditions with different tuning systems or modal frameworks.

Timbre Similarity ($S_{\mathbf{\text{timbre}}}$): Combines MFCC coefficient distributions (emphasizing lower‑order coefficients), temporal dynamics through delta features, and spectral shape characteristics. Statistical distributions preserve temporal complexity while avoiding information loss from simple averaging.

We acknowledge several methodological considerations that may affect our signal‑processing features’ cross‑cultural applicability. Our melody features rely on PYIN’s monophonic F0 extraction, which tracks the most salient pitch trajectory in polyphonic music but may miss actual melodic lines. Harmony features employ Western‑centric chord templates and Krumhansl–Schmuckler key profiles developed for tonal music, potentially inadequately capturing modal systems or non‑tertian structures. Timbre features emphasize MFCC‑based descriptors, which, while widely used, reflect specific assumptions about perceptually relevant spectral characteristics. While these approaches represent established practices in MIR research, their effectiveness across diverse musical traditions remains an open question. These design choices should be considered when interpreting signal‑processing performance relative to foundation models in our cross‑cultural evaluation.

The feature combination weights within each dimension reflect the relative contribution of each component to the overall similarity score (e.g., interval patterns or pitch class distributions for melody) and are informed by established music cognition research (McDermott, 2004). For melody, we slightly emphasize interval patterns (40%), while allocating equal weight to pitch class similarities (30% total, divided equally between semitone and quarter‑tone resolutions) and F0 statistical characteristics (30%). Rhythm features receive equal weighting (25% each) as tempo, onset patterns, beat intervals, and tempogram profiles provide complementary perspectives on rhythmic structure. Harmony components are equally weighted (20% each) to balance contributions from chroma features, chord analysis, key profiles, and tonal centroids. For timbre, we emphasize MFCC‑based features (90% total: 45% for coefficient distributions, 45% for temporal dynamics) over spectral shape features (10%), as MFCCs have demonstrated effectiveness in capturing timbral qualities (Tzanetakis and Cook, 2002).

These four similarity scores ($S_{\text{melody}},S_{\text{rhythm}},S_{\text{harmony}},$ $S_{\text{timbre}}$) serve as the independent variables in our linear regression analysis (Section 7.3.1), enabling systematic evaluation of how well each musical dimension aligns with human cross‑cultural similarity perception.

6.1 Representation extraction and processing

Foundation models require careful extraction of embeddings from their learned representations to obtain effective similarity measures. Following established best practices (Papaioannou et al., 2025; Plachouras, 2023), we employ model‑specific configurations optimized for music understanding tasks:

MERT models (MERT‑95M, MERT‑330M, CultureMERT variants) use 30‑s audio windows at a 24 kHz sampling rate. Representations are extracted by averaging hidden states across middle layers (layers 4–7 for MERT‑95M; layers 10–13 for MERT‑330M), then computing temporal means. This middle‑layer strategy balances low‑level acoustic detail with high‑level musical abstractions (Plachouras, 2023).

CLAP models employ 10‑s windows at 48 kHz to preserve spectral detail for diverse musical timbres. We extract representations from the audio projection layer that processes average‑pooled final hidden states, leveraging the model’s contrastive audio‑text alignment (Papaioannou et al., 2025).

Qwen2‑Audio uses 30‑s windows at 16 kHz, prioritizing extended temporal context for musical structure analysis. We extract representations by averaging last hidden state embeddings across all audio tower layers, using minimal text prompts to activate audio understanding while minimizing text modality biases (Papaioannou et al., 2025).

While alternative layer selections or pooling strategies could be explored, our choices follow published best practices (Papaioannou et al., 2025; Plachouras, 2023) that have been validated for music similarity and understanding tasks across diverse MIR settings.

For each foundation model, we compute pairwise similarity between audio representations using cosine similarity. Specifically, given foundation model representations ${\mathbf{r}}_1,{\mathbf{r}}_2\in {\mathbb{R}}^d$ for two audio clips, where $d$ is the model‑specific embedding dimensionality and $||\cdot |{|}_2$ denotes the L2 (Euclidean) norm, similarity is computed as:

This similarity measure is computed for all audio pairs of the dataset in the learned embedding space, enabling comparison with human annotations across the three similarity dimensions (overall musical, cultural, and recommendation‑level).

7.1 Data preparation and normalization

Our experimental framework addresses individual rating scale differences and handles outliers through a two‑stage normalization process designed to ensure fair comparison across computational methods and human judgments.

7.2 Evaluation metrics

We employ a comprehensive evaluation framework using five complementary metrics, each capturing different aspects of similarity alignment between computational methods and human perception (Tian et al., 2025).

Kendall’s Tau ($\mathit{\tau }$): Measures rank correlation using concordant and discordant pairs, providing a robust estimate of ranking agreement that is less sensitive to outliers:

where $n_c$ and $n_d$ represent the number of concordant and discordant pairs, respectively, and $n$ is the total number of observations (Kendall, 1938).

Spearman Rank Correlation ($\mathit{\rho }$): Evaluates monotonic relationships by comparing rank orderings, making this metric robust to non‑linear transformations of similarity scales:

where $d_i$ represents the difference between ranks for the $i$‑th observation (Spearman, 1904).

Normalized Discounted Cumulative Gain (NDCG): Adapted from information retrieval, NDCG evaluates the quality of similarity rankings by emphasizing correct identification of highly similar pairs. For each query audio, we rank all other clips according to computational similarity and compare against human similarity as relevance scores:

where ${\mathit{\text{rel}}}_i$ is the relevance of the item at position $i$ (computational method similarity) and ${\mathit{\text{rel}}}_i^{\ast }$ represents the ideal ranking (human similarity) (Järvelin and Kekäläinen, 2002).

Triplet Agreement: Evaluates whether computational methods preserve relative similarity orderings within triplets of audio clips. For each triplet $(a,b,c)$ where human judgments indicate $s_{\mathit{\text{human}}}(a,b)>s_{\mathit{\text{human}}}(a,c)$ by a margin $ϵ=0.1$, we assess whether the computational method produces the same ordering:

where $\mathrm{\Delta }{s}_{\mathit{\text{comp}}}=s_{\mathit{\text{comp}}}(a,b)-s_{\mathit{\text{comp}}}(a,c)$ and $\mathrm{\Delta }{s}_{\mathit{\text{human}}}=s_{\mathit{\text{human}}}(a,b)-s_{\mathit{\text{human}}}(a,c)$ represent the similarity differences for computational and human judgments, respectively; the indicator function $\mathbf{1}[\cdot ]$ returns 1 when the condition holds (both methods agree on ranking) and 0 otherwise; and $N$ is the total number of valid triplets. This metric is particularly relevant for retrieval applications where relative ranking matters more than absolute similarity values (Radinsky and Ailon, 2011).

7.3 Feature contribution analysis

To understand the relative importance of different musical dimensions in predicting human perception and foundation model behavior, we employ linear regression analysis using signal‑processing features as predictors.

7.4 Ensemble methods

To investigate the potential for combining signal‑processing features and foundation model representations for improved human similarity prediction, we implement ensemble regression using both linear and gradient boosting approaches. We follow the same multiple random split strategy as in the feature contribution analysis, performing five iterations with different random partitions of the data into training (80%) and testing (20%) sets to ensure robust performance estimates.

Linear Regression: Serves as a baseline ensemble approach, combining all available signal‑processing and foundation model features in a single linear model (Weisberg, 2005). Feature importance is directly interpretable through regression coefficients, enabling analysis of which feature combinations contribute most effectively to human similarity prediction.

LightGBM: A gradient‑boosting framework that can capture non‑linear relationships between features and targets (Ke et al., 2017). We employ early stopping with validation monitoring to prevent overfitting, using parameters optimized for regression tasks, including a learning rate of 0.05 and maximum of 1000 boosting rounds with early stopping after 50 rounds without improvement.

Ensemble models are, in turn, evaluated using the same comprehensive metric suite as individual computational methods (Kendall’s Tau, Spearman correlation, NDCG, and Triplet Agreement), enabling direct comparison of improvement over single‑method approaches. Performance metrics are averaged across the five random split iterations to provide robust estimates with confidence intervals.

7.5 Cross‑Cultural analysis framework

To analyze cross‑cultural patterns, we aggregate individual pair similarities at the dataset level by computing the mean similarity score across all annotated pairs belonging to each combination of two datasets. This produces $9\times 9$ similarity matrices representing relationships between musical traditions from both human and computational perspectives.

These dataset‑level similarities provide a macro‑level view of cultural relationships. For computational methods, we also evaluate cultural discrimination using the complete $468\times 468$ similarity matrix (approximately 100,000 unique pairs), which provides a more comprehensive assessment of cross‑cultural patterns beyond the human‑annotated subset.

We analyze within‑tradition versus between‑tradition discrimination using a distance‑based separation ratio that measures how well computational methods distinguish between different musical cultures.

8.1 Human user study results

8.1.2 Cross‑Cultural dataset relationships

Figure 1 presents the dataset‑level cultural similarity matrix, revealing systematic patterns in how participants perceive relationships between different musical traditions. We focus on cultural similarity as it most directly captures participants’ perception of tradition‑based relationships, distinct from purely musical or personal preference considerations. Several notable clusters emerge from the data:

Figure 1

Indian Classical Music Cluster: Hindustani and Carnatic traditions show the highest mutual similarity (0.69), reflecting their shared historical and theoretical foundations. Both traditions also show moderate similarity with other non‑Western traditions.

Mediterranean and Middle Eastern Music Cluster: Arab‑Andalusian, Lyra (Greek), and Turkish‑makam exhibit elevated mutual similarities, with Arab‑Andalusian showing particularly strong connections to Turkish‑makam (0.65). This clustering reflects historical cultural exchanges across the Mediterranean and Middle East regions.

Western Music Separation: MagnaTagATune and FMA‑medium show high mutual similarity (0.49) and generally lower similarities with traditional music datasets, suggesting that participants clearly distinguish Western styles from traditional world music. It is important to note that these Western datasets encompass internal heterogeneity, reflected in the relatively large circles for these datasets in the multidimensional scaling (MDS) visualization (Figure 2), indicating high within‑dataset diversity.

Figure 2

Chinese Opera and Flamenco Distinctiveness: Jingju (Beijing Opera) shows the highest within‑tradition similarity (0.91) and generally lower cross‑cultural similarities, indicating its unique characteristics that participants found difficult to relate to other traditions. Similarly, corpusCOFLA (Flamenco) demonstrates high within‑tradition similarity (0.75) while showing substantially lower cross‑cultural similarities (≤0.40), reflecting its distinctive musical characteristics that participants perceived as culturally distinct from other traditions.

These patterns align with musicological understanding of cultural relationships and historical influences, validating the meaningfulness of human cross‑cultural similarity judgments.

8.2 Signal‑Processing features and foundation models vs. human perception

We systematically evaluated both signal‑processing features and foundation models against human similarity judgments using five complementary metrics across the three similarity dimensions. This comprehensive evaluation provides insights into which computational approaches best align with human cross‑cultural music perception.

8.2.2 Comparative analysis across methods

Figure 3 provides a radar plot comparison of the top‑performing methods, selected based on highest average rank across all five evaluation metrics and three similarity dimensions. The eight methods shown (CLAP‑Music&Speech, Melody, MERT‑95, MERT‑330, CultureMERT, Qwen2‑Audio, CultureMERT‑TA, and Timbre) represent the best overall performers in our evaluation, sorted in descending order. The radar plot, averaged across similarity dimensions and normalized for visualization, clearly illustrates CLAP‑Music&Speech’s superior performance across most metrics, particularly excelling in Triplet Agreement and NDCG measures. Melody emerges as the strongest signal‑processing feature, showing balanced performance across all evaluation dimensions. The plot also reveals that foundation models generally maintain more consistent performance profiles compared to signal‑processing features, which show greater variability across different metrics.

Figure 3

We note that our evaluation metrics capture different aspects of similarity alignment: correlation‑based metrics (Spearman $\rho$, Kendall $\tau$) measure monotonic relationships and ranking agreement, threshold‑based metrics (Triplet Agreement, NDCG) assess retrieval quality, and MAE quantifies absolute error magnitude. These metrics may not be directly comparable in absolute terms; however, the consistency of relative method rankings across all five metrics, with CLAP‑Music&Speech and melody maintaining their leading positions, suggests robust performance patterns. The correlation‑based metrics show low absolute values across all methods, reflecting the inherent challenges in capturing the complex, non‑linear relationships that characterize human cross‑cultural music similarity perception. These results demonstrate that, while foundation models achieve superior alignment with human perception, the gap between computational methods and human judgment remains substantial, indicating significant room for improvement in cross‑cultural music similarity modeling.

8.3 Cross‑Cultural discriminability analysis

To assess how well computational methods distinguish between different musical traditions, we analyze their cross‑dataset discriminability using distance‑based separation ratios.

Table 6 presents the distance‑based separation ratios for humans and all computational methods. The results reveal significant differences in cultural‑discrimination capabilities. Human similarity judgments demonstrate superior cultural discrimination, with the cultural similarity dimension achieving the highest separation ratio (2.361), followed by recommendation‑level (2.106) and overall musical similarity (1.803).

Among signal‑processing features, melody emerges as the most discriminative (1.276), while rhythm, harmony, and timbre show limited discrimination capabilities with ratios close to 1.0. Foundation models substantially outperform signal‑processing features, but reveal an important trade‑off between universal musical understanding and cultural discriminability. Qwen2‑Audio achieves the highest separation ratios among all computational methods (1.579 for annotated pairs, 1.602 for all pairs), demonstrating superior ability to distinguish between musical traditions. In contrast, CLAP‑Music&Speech, while excelling at human similarity alignment, shows more modest discrimination performance (1.366), suggesting that models optimized for universal cross‑cultural musical understanding may sacrifice some discriminative power between cultural boundaries.

8.4 Feature‑Contribution analysis

To understand which musical dimensions drive similarity perception in both human judgments and foundation model representations, we performed linear regression analysis using signal‑processing features as predictors. This interpretability analysis reveals the relative importance of melody, rhythm, harmony, and timbre in explaining similarity patterns across different evaluation contexts.

Figure 4 presents the linear regression weights when predicting human similarity judgments and foundation model similarities using the four signal‑processing features. The analysis reveals distinct patterns in how different targets weight various musical dimensions, providing insights into the underlying factors that drive similarity perception.

Figure 4

8.5 Ensemble methods for human similarity prediction

To leverage the complementary strengths of signal‑processing features and foundation model representations, we developed ensemble regression methods that combine all computational approaches to predict human similarity judgments. This analysis explores whether integrating interpretable musical features with learned representations can achieve superior alignment with human cross‑cultural music perception.

Table 7 presents the performance of both linear regression and LightGBM ensemble methods across all evaluation metrics and similarity dimensions. The ensemble methods outperform individual approaches, as shown in Table 5. Linear regression ensemble achieves triplet agreement scores of 65.1%–67.0% compared to the best individual method (CLAP‑Music&Speech) at 62.6%–64.9%. Similarly, NDCG scores reach 90.9%–92.5% versus the previous best of 88.0%–89.8%. The correlation metrics also show consistent gains, with Spearman correlations reaching 0.15–0.19 compared to individual method maximums of 0.11–0.16.

Most significantly, the ensemble methods achieve substantial reductions in prediction error, with MAE values of 19.7%–23.2% representing improvements of approximately 8–9 percentage points over the best individual methods (melody features at 29.5%–30.9% MAE). This represents a relative error reduction of roughly 25%–30%, demonstrating that combining multiple computational approaches provides complementary information for predicting human similarity judgments. Detailed contribution analysis is provided in the supplementary material.

8.6 Validation of perceptual stability

A critical methodological concern in our study is whether the observed correlations between human judgments and computational methods represent stable perceptual patterns rather than noise, given that each audio pair receives approximately one rating (Section 4). To address this concern, we conducted random permutation tests for all computational methods across all three similarity dimensions.

For each combination of human dimension and computational method, we randomly shuffled human ratings among the 1,130 annotated pairs 10,000 times and recomputed Spearman and Kendall correlation coefficients. If computational similarities predicted human ratings no better than chance, permuted correlations would equal or exceed observed correlations approximately 50% of the time.

Results demonstrate that most computational methods achieve correlations significantly above chance levels. Of the 33 method‑dimension combinations tested, 28 show significance at $p<0.001$ for both Spearman $\rho$ and Kendall $\tau$. The five exceptions include: harmony features across all three human dimensions (Spearman $p>0.22$), consistent with our earlier discussion of Western‑centric assumptions in harmonic feature design (Section 5), and rhythm features for cultural similarity ($p=0.019$) and recommendation‑level similarity ($p=0.047$). Notably, all foundation models achieved highly significant correlations ($p<0.001$) across all dimensions, as did melody and timbre features.

These findings, combined with: (1) systematic patterns in dataset‑level similarity aggregations showing interpretable cultural relationships (Section 8.1.2), (2) high inter‑dimensional correlation consistency suggesting coherent underlying similarity concepts across participants, and (3) interpretable MDS clustering revealing meaningful cultural groupings (Figure 2), provide converging evidence that stable cross‑cultural similarity factors exist despite approximately single‑rater annotations per pair, modest absolute correlation values, and potential individual variation in similarity criteria.