Predicting Eurovision Song Contest Results: A Hit Song Science Approach

2.1 Rules of the eurovision song contest

Participating countries in the Eurovision Song Contest select an artist or group to represent them, who will then perform an original song that may be of any style or genre, provided it includes a vocal component (Stockemer et al., 2017). While there are no restrictions on the language of the lyrics, participants frequently opt to sing in either English or their native tongue (Haan et al., 2005).

Eurovision is divided into two stages, with two semi‑finals and a grand final. After each semi‑final performance, viewers vote for their favourite performances; however, they are not permitted to vote for the song representing their home country. Each semi‑final’s top 10 songs advance to the grand final. Furthermore, a few countries automatically qualify for the grand final without having to take part in the semi‑finals. These include the ‘Big Five’ (France, Germany, Italy, Spain, and the United Kingdom) and the host nation, which is typically the previous year’s winner (Haan et al., 2005). Since 2023, the semi‑final voting has been entirely public.

In the grand final, the voting mechanism includes an international public vote and national juries, each awarding points to 10 countries; these scores are tabulated separately and then added together. The winner is the song with the most overall points. The national juries award points as follows: a song deemed to be a country’s ‘favourite’ receives 12 points, the second‑favourite receives 10 points, and the remaining songs are awarded 8 to 1 point. Songs that are not in the top 10 receive 0 points from that country (Haan et al., 2005).

2.2 Related work

A recent paper introduced a comprehensive dataset combining contest metadata, historical voting data, betting odds, and audio features extracted from recorded performances. To create a stable measure of competitiveness, the authors converted raw contest scores into an interval‑scale metric called ‘cantobels’, allowing for comparisons across years. The study applied music‑understanding models to extract audio embeddings from each song’s recorded performance and employed learning models to simulate and predict contest outcomes. The results demonstrated that these audio embeddings accounted for 7.2% of the variance in the Eurovision competitiveness measure. The authors emphasised that while this percentage is relatively small, it remains meaningful, highlighting the potential of the Eurovision Song Contest as a testing ground for Music Information Retrieval (MIR) algorithms (Burgoyne et al., 2023).

Monitoring competition ‘buzz’ through press attention, fan feedback, and social platforms can gauge a song’s popularity. An analysis of 900,000 English and Spanish tweets from the 2019 Eurovision final identified overall Twitter sentiment and linked tweets to key event moments (Demergis, 2019). Excluding national pride tweets, positive tweets for each country were counted to produce a predicted ranking. Using an emoticon‑derived sentiment lexicon and a Naive Bayes model, it was found that top televised performances received more positive tweets. The study compared the predicted and actual rankings using Spearman’s and Kendall’s rank correlation coefficients, achieving 0.559 and 0.403, respectively. Another study with 1 million tweets achieved coefficients of 0.62 and 0.72 using sentiment lexicon‑based and deep learning approaches (Kumpulainen et al., 2020).

2.3 Voting patterns

Research in 1996 has already shown that biased evaluations in Eurovision voting undermine equal chances for all participating nations, with systemic preferences forming among European blocs (Yair and Maman, 1996). Although new rules, such as limiting the presence of neighbouring countries in the same semi‑final, aimed to reduce such biases (Svete et al., 2020), studies indicate that they continue to persist. Historical voting patterns reveal the existence of alliances and preferences shaped by cultural, linguistic, and political connections among nations.

Analysis of Eurovision data from 1998 to 2008 identified key factors driving favouritism, including religion, language, geographical proximity, shared borders, semi‑final participation, GDP disparities, and immigration (Luković et al., 2019). ‘Patriotic voting’, where migrants vote for their home country, highlights cultural attachments. In addition, affluent nations often vote for economically poorer and culturally distinct countries, while poorer regions favour nations with similar demographics and cultures. In addition to geographical, cultural, and socio‑economic factors (Blangiardo and Baio, 2014; Dekker, 2007; Ochoa et al., 2008), one‑way relationships (Svete et al., 2020) and exchange partnerships (Mantzaris et al., 2018) have also been identified.

A 2007 study concluded that collusive voting stems from quasi‑political motivations rather than shared musical tastes (Gatherer, 2007). While some nations promote regional musical styles, songs from wholly unrelated genres are also regularly introduced. The study also highlighted the impact of familiarity, as exposure resulting from semi‑final success often boosts voting outcomes in the final.

2.4 Order effects

The order of appearance of participants in the competition is random; yet it has been found that it can influence the final results. Contestants who perform towards the end of the event tend to achieve higher ranks (Antipov and Pokryshevskaya, 2017; Arroyo‑Barrigüete et al., 2022; Ginsburgh and Moreno‑Ternero, 2023; Haan et al., 2005). Moreover, a relatively strong performance is likely to score more points if it is surrounded by poorer performances (Ginsburgh and Moreno‑Ternero, 2023).

A 2022 study examining voting outcomes from 2014 to 2019 concludes that the order of performance can have a substantial impact on decision‑making, particularly in public voting (Arroyo‑Barrigüete et al., 2022). To mitigate this order bias, the paper calls for expert judges who can evaluate performances more objectively. This is in agreement with a 2005 investigation of the difference between expert and audience voting. It contends that juries are less influenced by external factors, such as the performance order, when evaluating performances (Haan et al., 2005).

2.5 Voting motivation

A 2016 Eurovision survey investigated voting behaviours and influences (Stockemer et al., 2017). An online questionnaire with over 500 responses revealed that 26% voted based on personal preference, while another 26% voted for the song they thought would win. Strategic voting was employed by 11% to select the likely winner among favourites. In addition, language affinity, ethnic similarities, positive sentiments, bloc voting, political concerns, and cultural expectations accounted for 37% of the votes. The survey indicates that people may vote for likely winners over personal favourites, reflecting popular sentiment and contestant recognition. This highlights that a song’s appeal extends beyond its musical merits, which is crucial for predicting contest results.

2.6 Multi‑Modal approaches to HSS

The scope of modalities considered for predicting hit songs has expanded significantly over time. Early efforts focused on feature sets and high‑level audio features (Pachet and Roy, 2008), while more recent approaches have incorporated lyrics analysis (Raza and Nanath, 2020) and the integration of diverse extrinsic factors—elements that may directly or indirectly influence musical success (Seufitelli et al., 2023). The literature highlights the advantages of supplementing high‑level audio features with attributes such as artist collaboration networks (Silva et al., 2022) and listener behaviour data (Herremans and Bergmans, 2017).

Including the release year as a feature has also proven to enhance song popularity prediction by capturing temporal musical trends (Zangerle et al., 2019). In an experiment classifying songs as hits or non‑hits using both low‑level and high‑level audio features from the Essentia toolkit (Bogdanov et al., 2013), the release year improved model performance by providing essential temporal context.

A notable study employing an end‑to‑end multi‑modal deep learning architecture demonstrated that combining audio, lyrics, and metadata significantly enhanced the model’s predictive accuracy (Martin‑Gutiérrez et al., 2020). The audio features included high‑level attributes extracted from Spotify and low‑level features derived from audio representations such as Mel‑spectrograms, Chroma features, and spectral centroids. Lyrics were analysed for factors such as vocabulary richness and sentence‑similarity coefficients, while the metadata incorporated metrics such as artist popularity, follower count, and the number of markets where the artist’s music was available.

2.7 Considerations

Predicting Eurovision results requires a multi‑modal approach. From an HSS perspective, incorporating contextual attributes beyond audio and lyrics modalities is essential for improving model performance. In Eurovision, certain variables are known to influence a song’s success, including performance order, voting patterns between countries, and the familiarity gained from semi‑final appearances or pre‑contest exposure. Accounting for these factors is crucial for effectively contextualising songs within the competition’s framework.

This task also presents inherent challenges, including the evolving preferences of judges and audiences and the competition’s changing format. Political events throughout the year can influence voting behaviour, leading countries to support or oppose one another. In addition, the competition showcases a wide variety of genres and styles—from pop and rock to traditional regional sounds—which differ significantly among participating countries and vary from year to year.

3.1 Dataset collection

This study encompasses contest data from 2008 to 2024, a period without major competition format changes. The dataset comprises 646 Eurovision songs, with data for the year 2020 excluded due to the cancellation of the contest in response to the global COVID‑19 pandemic.

The following metadata was collected: song artist and title, represented country, year, final contest placement, semi‑final placement, running order in both the semi‑finals and final, the specific semi‑final in which the song was performed, and the song lyrics. In addition, three YouTube links were collected for each song: (1) the studio recording, (2) the live performance, and (3) the official music video uploaded by the Eurovision Song Contest’s YouTube channel⁵ prior to the competition. Songs were labelled based on whether they qualified for the final, did not qualify, or were pre‑qualified as part of the Big Five.

3.2 Features

3.3 Prediction framework

Developing a framework for predicting Eurovision results involved accounting for the contest format, as not all participating songs compete in the same event. Therefore, the dataset was adjusted into two versions: the semi‑finals dataset and the grand final dataset, which included only songs that qualified from the semi‑finals and those that pre‑qualified. This is shown in Figure 2. The semi‑finals dataset included 554 songs, while the grand final dataset included 411 songs, covering contests between 2008 and 2024.

Figure 2

For each dataset, a hold‑one‑out approach was employed to conduct multiple experiments. In each experiment, a different year from 2008 to 2024 was designated as the test year, with all other available years used as training data. This strategy aims to produce more statistically meaningful results and insights that can be generalised across contests, rather than focusing solely on the rankings of a single recent contest. When interpreting the results of a model tested on only one year, it is unclear whether the performance reflects the model’s overall ability to estimate contest results or if it is influenced by that year being an outlier. While it may appear that some future events are used to predict the past, this approach assumes that patterns learnt across multiple years—such as voting behaviours, public appeal trends, and musical characteristics—remain relevant.

For the grand final dataset, the model was tested on the final ranking of the selected contest year. For the semi‑final dataset, two test sets were used per year, corresponding to the rankings of both semi‑finals.

The study experimented with all possible combinations of feature sets, proposing seven approaches, outlined in Table 2.

By comparing the differences in predictions between these feature sets, the study evaluates the relevance of each feature group as predictors and accounts for any differences that may arise, particularly when contrasting models that include YouTube daily views with those that do not.

4.1 Exploratory analysis

Correlation analysis using Spearman’s correlation, with scaled ranking positions from the semi‑finals and grand finals as target variables, revealed that the attribute most strongly correlated with Eurovision rankings is YouTube daily views, with correlation coefficients of (–0.34) for the semi‑finals and (–0.29) for the final rankings. Semi‑final rankings also correlate with the previous year’s final vote reciprocation (–0.29), final vote ratio (–0.28), and running order of performances (–0.21). Grand final rankings have a lower correlation with the running order (–0.16) and the previous year’s final vote reciprocation (–0.16). The negative correlations indicate that higher YouTube daily views, better previous results, and later performance slots are associated with higher rankings in the contest.

Correlation coefficients between audio features and Eurovision rankings are very low, with dynamic complexity showing (–0.11) for semi‑finals and (–0.16) for grand final rankings, the 1st principal component of ERB bands features at (–0.13) for semi‑finals and (–0.14) for grand final rankings, and average loudness at (0.11) for semi‑finals and (0.13) for grand final rankings. Dynamic complexity, which measures fluctuations in loudness, and certain spectral characteristics may slightly influence rankings, whereas the weak positive correlation with average loudness suggests that louder performances might have a small tendency to rank higher.

Interestingly, audio features exhibit more significant correlations with YouTube daily views than with Eurovision rankings. For the songs in the semi‑finals, YouTube daily views correlate with the 1st principal component of Mel bands features (0.43), the 1st principal component of Bark bands features (0.41), and the 2nd principal component of spectral features (–0.32). For the songs in the grand finals, the correlation coefficients are (0.42), (0.40), and (–0.30), respectively. These correlations suggest that some spectral characteristics, particularly those related to perceived pitch and timbre, may be related to increased public appeal. Since both Mel and Bark scales are designed to approximate human auditory perception, this could indicate that songs with particular harmonic content or specific frequency distributions are more likely to resonate with audiences. This suggests a relationship between song qualities and public appeal that is not related to contest success.

In addition, computing the vote reciprocation rate for participating countries revealed relevant voting relationships, such as the Nordic voting bloc (Sweden, Norway, Finland, Denmark, and Iceland) and the bloc encompassing Ukraine, Azerbaijan, and Russia. Other notable pairs with frequent vote exchanges include Armenia–France, Greece–Azerbaijan, Moldova–Ukraine–Romania, Israel– France, and Armenia–Russia. These voting relationships reflect geographical, linguistic, and cultural proximities, as discussed in the literature.

4.2 Experimental setup

XGBoost was selected as the regression model for its robustness, efficiency, and strong performance in handling structured datasets (Chen and Guestrin, 2016). To further optimise the model, AutoML for XGBoost from the Fast and Lightweight AutoML (FLAML) library⁷ was implemented in Python. FLAML was chosen for its lightweight and efficient approach to automated machine learning, which minimises the computational cost of hyperparameter tuning while maintaining high model performance.

For each experiment and test year, PCA was applied separately to the train and test sets for all feature sets containing audio features, ensuring no data leakage. Afterward, each feature set was normalised, and the XGBoost model was tuned using the R² metric as the primary evaluation metric with a time budget of 120 seconds. The tuned models were saved, and predictions were subsequently made on the respective test sets.

The target variables, which represent the ranking positions achieved by songs in the semi‑finals and grand final, have been standardised between 0 and 1 in the training data, with 0 corresponding to the highest‑ranked song and 1 corresponding to the lowest‑ranked song. The regression models produced predictions within the same range. These values were then sorted from smallest to largest to create a predicted Eurovision ranking. This methodology was applied to each event: semi‑final 1, semi‑final 2, and the grand final. Typically, 15– to 19 countries compete in eacha semi‑final, and 25– to 27 participate in the grand final, including the automatic qualifiers.

The regression results were interpreted using the R² metric to understand how well the data fit the model. R² measures the proportion of variance in the dependent variable that is predictable from the independent variables.

The predicted rankings of the Eurovision semi‑finals and grand final were evaluated using Spearman’s () and Kendall’s () rank correlation coefficients. These coefficients were employed to assess the similarity in the ordering of the rankings by determining how closely they resembled the actual results (Demergis, 2019).

Average rank errors were also computed for each predicted ranking to evaluate the accuracy of predictions by measuring the average deviation between the true rankings and predicted rankings. In practice, an average rank error of 0 signifies perfect predictions, while higher values reflect larger discrepancies between predicted and actual ranks, indicating less accurate predictions.

The results are summarised in Tables 3–6 and Figure 3. Table 3 presents the average rank errors across all experiments for the semi‑finals and grand finals. For the semi‑finals, averages are computed first across both events (semi‑final 1 and semi‑final 2) within each year and then across all years. Table 4 reports Spearman’s rank correlation coefficients for all feature set approaches across all years of semi‑finals, with separate values for the first and second semi‑finals. Only coefficients with statistically significant correlations (p‑value < 0.05) are included. A dash in these cells indicates that the coefficient was not statistically significant. Figure 3 displays the average rank errors, while Table 5 provides the test values for all feature sets across all experiments using grand final data. Finally, Table 6 presents Spearman’s rank correlation coefficients for all feature set approaches across all years of grand final data, again including only significant correlations (p‑value < 0.05).

Figure 3

4.3 Approach 1: Intrinsic characteristics of songs

On average, models trained on audio and lyrics features yielded the highest average rank errors for both the semi‑finals and grand final, making this the least accurate approach. The very low R² values in Table 5 indicate that these models struggle to capture the relationship between intrinsic song attributes and final rankings. An examination of the ordinal correlations in Tables 4 and 6 reveals that the predictions generally did not align with the true rankings. The observed correlations for the 2022 first semi‑final and the 2015 grand final appear to be outliers.

4.4 Approach 2: Intrinsic characteristics of songs and eurovision data

While the incorporation of Eurovision contest data slightly reduced the average rank errors, the models still struggled to estimate rankings accurately, as indicated by the R² values in Table 5. However, when examining the results for the earliest years in the dataset, 2008 and 2009, the improvement was more substantial. This is reflected in Spearman’s ordinal correlations for the grand finals, which were (0.42) and (0.59), respectively. A similar trend was observed for 2008’s first semi‑final and both semi‑finals in 2009. This pattern, however, was not observed in the later years, suggesting that contest‑specific data may have been more relevant in the past.

4.5 Approach 3: Intrinsic characteristics of songs and public appeal

The inclusion of a feature set combining intrinsic song features and YouTube daily views further improved predictions by reducing average rank errors. This approach often yielded moderate‑to‑strong Spearman’s correlations, particularly in the semi‑finals (Table 4), where audience engagement plays a significant role. For the grand finals, model performance, as represented by R², improved, indicating a better data fit. Significant ordinal correlations were achieved in 10 out of the 16 years tested, with 2013 and 2015 outperforming other approaches. The public appeal feature, represented by YouTube daily views, emerges as a critical factor for enhancing prediction accuracy.

4.6 Approach 4: Intrinsic characteristics of songs and public appeal and eurovision data

This approach, which combines all available features, further reduced average rank errors in both the semi‑finals (3.70) and grand finals (6.07) (Table 3). Spearman’s rank correlations were significant in most years for both the semi‑finals and the grand final, often outperforming other approaches—particularly in the grand finals of 2024, 2021, 2019, 2017, and 2009. Integrating intrinsic song characteristics, public appeal, and contest‑specific data provides a promising prediction model which performs relatively consistently across semi‑finals and grand finals over the years.

4.7 Approach 5: Public appeal

As the YouTube daily views metric has been identified as a prominent predictor of Eurovision success, it is important to evaluate how well this feature performs independently, without intrinsic song features or contest data.

On average, this approach resulted in higher average rank errors in the semi‑finals (4.08) compared to Approach 4, which combines all features. It also produced fewer ordinal correlations across both semi‑finals, indicating less consistent predictions across the two events each year. However, in some cases, the correlations for the first semi‑finals were exceptionally high, such as (0.95) in 2014, (0.82) in 2015, (0.92) in 2019, and (0.85) in 2023, outperforming other approaches in those years.

Focusing on semi‑final predictions for 2023 and 2024, the first two years when rankings were determined entirely by public vote, Approach 4, which incorporates all features, achieves rankings that significantly correlate with actual results for both semi‑finals. However, this approach does not achieve significant correlations for the second semi‑finals (Table 4).

According to the average rank errors for grand finals (Figure 3), this approach outperformed others between 2011 and 2017, achieving the lowest average rank error across all years (6.05). However, in grand final predictions, this approach only outperformed others in terms of Spearman’s rank correlation in 2012, 2014, and 2016, falling short compared to approaches that combine public appeal with additional features.

While public appeal alone can be a strong predictor in some cases, its inconsistency makes it less reliable. The post‑contest exposure effect plays a significant role; while more successful songs often attract greater attention, as reflected in daily view counts, popularity can also stem from factors unrelated to contest performance. Moreover, YouTube daily views might be insufficient for grand final ranking predictions, where jury votes play a significant role alongside public voting.

4.8 Approach 6: Eurovision data

This approach showed reduced performance across all metrics compared to Approaches 3, 4, and 5. Although useful for understanding Eurovision‑specific dynamics, relying solely on performance order and country voting patterns proved insufficient for accurate predictions.

When compared to Approach 2, which combines intrinsic song features with contest data, the limited number of ordinal correlations, particularly for the second semi‑final across multiple years, underscores the importance of historical voting patterns.

4.9 Approach 7: Public appeal and eurovision data

This approach, which excludes intrinsic song features, ranked among the top‑performing methods, particularly in the semi‑finals, achieving the lowest average rank error (3.47). It also demonstrated the most consistent ordinal correlations, predicting both semi‑finals in 8 out of 16 years with moderate‑to‑strong correlations.

However, its performance on grand final data was weaker than that of other approaches. This is evident in its average rank error (6.27), R² values (Table 5), and fewer ordinal correlations, outperforming other feature approaches only in 2008. Combining public appeal with Eurovision data offers a robust model, particularly in the semi‑finals, but it remains insufficient for grand final predictions.

Baseline comparison: Random rankings vs. model predictions

4.10

Random rankings for both semi‑finals and grand finals were generated in a 1,000‑run simulation, considering the contest years in the dataset and the number of participating songs each year. These random rankings serve as a baseline to compare against predictions, assessing how well the models perform relative to chance and how effectively meaningful patterns are captured. The simulation produced an average rank error of (8.55) for the grand final, with a 95% confidence interval of (8.01, 9.08). For the semi‑finals, the average rank error was (5.76), with a 95% confidence interval of (5.44, 6.07).

Comparing the random rank errors with the average rank errors across all years in Table 3, predictions outperformed chance in all cases except Approach 1 for the semi‑finals. In Figure 3, which displays average rank errors across all years for grand final results, predictions of Approaches 3, 4, and 7 consistently performed better than random.