A Mobile Application–Based Citizen Science Product to Compile Bird Observations

Mobile phone application

The mobile phone application allows users to record bird sounds anonymously with observation location information, and get species identifications for the recordings. There are two different recording schemes: opportunistic and interval recording. Opportunistic recordings are at most 5-minute-long recordings, with which users can record any vocalizing bird or soundscape they want to identify. Interval recordings allow users to produce sound data through a more congruent sampling scheme. Interval recording period can be selected to be one, three, six, or twelve hours during which the application records one-minute clips followed by nine-minute pauses. Mobile application can ensure the phone’s battery sufficiency during interval recordings. If the battery charge is too low, the recording will be interrupted, and the saving will be secured. When sending the recording, the user is asked for permission to use the recording for scientific purposes (consent), and identification results containing the identified species and confidence level are returned to the user in real time. Users can evaluate the identification result on the application by indicating whether the identification is correct or not and if they have seen or heard the bird.

The user interface of the application is shown in Figure 1. In addition to making new recordings (panel a), users can view their previous recordings (panel b), see a list of all species they have observed (panel c), and inspect observations made by all users on a map that includes each bird, which has been classified with at least 90% confidence or confirmed by the user who made the recording (panel d). The application is available for free for both iPhone and Android users in Finland. Importantly, the user interface of the application was designed to be accessible for all kinds of users (i.e., for example, in terms of visual layout for people with poor eyesight).

Figure 1

When the initiative relies on technological developments to succeed, participation of specialists with suitable engineering skills is crucial. Managing large quantity datasets needs research infrastructure and staff, which in turn takes time, effort, resources, and coordination to capitalise on the potential for public engagement. The audio data and model outputs were thus stored centrally on CSC (Finnish IT Center for Science) servers. The users remain completely anonymous in the data collection process; however, the users can be contacted through the phone application by their anonymous user ID.

Advertising the application

The application was published in collaboration with the national public broadcasting company (i.e., the Finnish national broadcasting company, YLE), and thus it received significant attention in the media. The application was first publicly mentioned and used on April 12^th 2023 on Metsäradio (Forest Radio), a show that discusses forestry, nature, and outdoor life. The application was also mentioned in the Luontoilta (Nature Evening) radio broadcast on May 4^th. On May 10^th, the application was advertised in YLE’s special Muuttolintujen kevät, a television broadcast featuring migratory birds, and on the main news television broadcast on YLE. It is noteworthy that the average audience for YLE News on television is 751,000, which represents roughly 14% of the Finnish population. YLE’s social media channels also promoted the Spring of Migratory Birds campaign to citizens throughout the spring, which included providing general information about migratory birds and how people could use the application to follow their arrival. In addition, the application was mentioned in several local and national journals during the spring as well as in various bird-related channels on social media. Gamification was also incorporated into the application. Based on the number of species observations, the user was given virtual badges (three levels), which could be shared through the application on Instagram, Facebook, and X/Twitter.

Bird sound recognition model

The identification model is a convolutional neural network (CNN), which was constructed by using the BirdNET recognition model (Kahl et al. 2021) as a convolutional base and fine-tuning the network with vocalizations of Finnish bird species roughly following the pipeline presented previously (Lauha et al. 2022). The vocalizations of Finnish birds were obtained from the bird sound library Xeno-canto (Xeno-canto Foundation 2005) and from Finnish field recordings collected with autonomous recording units within the ongoing Lifeplan project (ERC-synergy project No. 856506) and annotated in the Bird Sounds Global portal (https://bsg.laji.fi), as well as from a previous pilot project (Lehikoinen et al. 2023) and targeted field recordings by Harry J. Lehto. The audio data were split into 3-second segments and converted to spectrogram images, which were used as training data for the CNN. Similarly, new recordings are analysed by splitting them into 3-second segments with 1-second overlap and predicting for each segment separately.

The identification model was originally trained for 150 Finnish bird species, and during the summer of 2023, the list was supplemented with ten more species. In the mobile phone application, the predictions of the model are adjusted by decreasing the predictions for species that are unlikely to occur in the recording based on latitude and time of the year. The application shows all species that are detected with at least 60% confidence.

Audio data

We analyse the audio data collected through the application between April 1^st, 2023 and July 31^st, 2023.The descriptive numbers of user volume and recordings are introduced in the results. We compared the mobile phone recordings with biomonitoring data collected with autonomous passive acoustic recorders from six sampling sites across Finland. The sampling is conducted as a part of the international Lifeplan project (ERC-synergy project No. 856506). The sampling scheme on Lifeplan sites corresponds to continuous interval recording in the application (i.e., audio samples of one minute are recorded once every ten minutes). We randomly selected one recording per hour from each site from the beginning of April until the end of July, which led to a data set of 15,900 recordings containing 21,200 species identifications with at least 60% certainty (1.3 species per recording).

Analysis methods

We compared the recordings made by users (i.e., application recordings) with PAM recordings by visualizing the spatial and temporal distributions of recordings and bird observations in both data sets. To analyse the locality of users, we defined a centre location for each user by selecting the median latitude $\overline{{\varphi }_u }$ and median longitude $\overline{{\lambda }_{\hspace{0.17em}u}}$ of all recordings by user u. We then calculated the distances between all of a user’s recording locations φ_j,u, λ_j,u and the centre location $\overline{{\varphi }_u},\hspace{0.17em}\hspace{0.17em}\overline{{\lambda }_u}$. We quantified the temporal distribution of recordings and bird observations by calculating the total number of recordings and the total number of species identifications on different confidence levels for each day. To analyse the activity of birds and application users around the clock, we calculated the total number of bird observations with at least 60% confidence for each hour of the day and divided it by the total number of recordings for both application and PAM recordings.

We analysed the user retention by dividing users to three groups based on the total number of recordings made (setting the thresholds at 50 and 500 recordings) and calculating the temporal distribution of recordings for each group. For each user u, we calculated the proportion of recordings ${\dot{r}}_{d,u }$ made in each day d by dividing the daily number of recordings r_d,u by the total number of recordings made by user until the last day of the sampling period $(d_u^{max}:\mathrm{31.7.2023}):{\dot{r}}_{(d,u)}=r_{(d,u)}÷{\Sigma }_{d=1}^{d_u^{max}}{r}_{d,u}$. We then calculated daily means of recording activity ${\dot{r}}_{d,u }$, for each group including all users who made their first recording at least d days before the end of the sampling period, and we visualized these average recording activities as a function of days since first recording.

The recording behaviour of active users was assessed by calculating species accumulation curves for all users with at least 200 recordings and all six PAM sites. The species accumulation curves were formed by calculating the cumulative number of unique species identified with at least 90% confidence as a function of the number of recordings made by the user.

We evaluated the quality of user annotations by selecting a sample of users and collecting a set of 50 annotations per user. These annotations were checked by an experienced bird specialist. Because user-annotated data can be used for improving the identification models (Kahl et al. 2021; Lauha et al. 2022), and instances where the model prediction is far from the true label are most useful for model improvement, we steered the data selection towards users who have generated the most annotations and data points in which the disagreement between the user and model prediction is greatest. To select the validation set, we calculated the number of positive annotations $n_u^P$ and number of negative annotations $n_u^N$ made by each user u and ranked the users by the geometric mean of $n_u^P$ and $n_u^N$: ${\overline{n}}_u=\sqrt{n_u^P\times n_u^N }$. Geometric mean was used to ensure that the data set contains both positive and negative annotations from each user. We then selected 50 users with the highest ${\overline{n}}_u$. For each user, we sorted the annotations according to the disagreement between the user annotation and model prediction: $d_{i,j}=\hspace{0.17em}|a_{i,j}-p_{i,j}|$, where a_i,j is the user annotation (true = 1 or false = 0) and p_i,j the model prediction for species i in recording j. Finally, for each user, we selected 50 annotations containing an equal amount of positive and negative annotations (if possible) from as many different species as possible starting from the highest values of d_i,j.

The user annotations are recording-specific, whereas model outputs are produced separately for several 3-second (s) clips within the recording. For each annotation, we selected the 3 s clip with the highest prediction for the target species including additional 0.5 s buffers before and after the clip to help the bird specialist to identify the correct species. The specialist listened through all 50 × 50 clips and noted if the target species was present in the recording.

How people welcomed the application

During a single season (April–July 2023), the application was downloaded more than 220,000 times, and more than 140,000 users submitted close to three million recordings (on average 21.3 recordings per user) of the Finnish birds. The median user submitted 8 recordings (2.5–97.5% quantiles: 1–122), while the most active user submitted more than 5,000 recordings, and 7.2% of users were responsible for 50% of the recordings. The recordings contain more than 5.8 million species identifications with at least 60% confidence (2.0 species per recording). Also, 1.2 million identifications (20.6%) were annotated by the users to be either true or false. A vast majority of the recordings were opportunistic, and only 10,300 recordings (0.35%) were interval recordings. The median recording duration was 26.5 s for opportunistic recordings (2.5–97.5% quantiles: 4.1–299.8 s) and 373.5 s for interval recordings (341.5–3881.5 s). It is noteworthy that even though the application was designed to classify Finnish birds and is available only for Finnish users, a small proportion (ca. 1%) of recordings were from outside of Finland. There were recordings from every continent except Antarctica.

Spatio-temporal distribution of recordings

The recording locations strongly follow the population density in Finland, indicating that people tend to go birdwatching close to their homes (Figure 2). Indeed, most of the recordings were done within a ca. 10 km radius from the centre point of individual user–produced spatial data (Figure 3c). The maximum recording distance was typically a few hundred kilometers from the individual centre location and almost 25% of active users (i.e., people who made more than 50 recordings) made recordings further than 300 km from their home locality (Figure 3e). A handful of users recorded audio more than 1,100 km away from their centre location, which is close to the longest possible distance between two locations within Finland and thus corresponds to a person recording both on the coast of Southern Finland and in Northern Lapland. The geographical coverage of recording locations is comprehensive, especially in relatively densely inhabited Southern Finland. In remote locations, the spatial coverage is predictably much lower. The sparse road network of Northern Finland is evident in the heatmap of recordings (Figure 2).

Figure 2

Figure 3

The recording behaviour of the users shows clear temporal patterns. Both recording activity and the number of species observed peaked for two weekends following May 10^th, 2023, when YLE launched a special event on migratory birds and the application was featured on national television. Throughout the summer, the recording activity was notably higher on weekends than weekdays (Figure 4a). Excluding the activity peaks in mid May and on weekends, the recording activity remained at a similar level from late May to early July, after which it diminished (Figure 4b). Figure 5, too, shows that the recording activity of users decreases over time.

Figure 4

Figure 5

Even though the recording activity of users depends on external factors such as the day of the week and the visibility of the application in the media, there is still a high and statistically very significant correlation (r = 0.749, p = 3.1e – 23) between the daily number of user recordings and the activity of birds (measured as daily species observations in PAM data), which indicates that people tend to produce more recordings when the birds are most active. However, within the day, this does not apply, and there is no statistically significant correlation between hourly number of user recordings and the vocalization activity of birds (r = 0.220, p = 0.302). Although bird activity peaked daily at approximately 5 a.m., people prefer to record at more convenient times and the hourly distribution of recordings peaked at approximately 10 a.m. and just before 8 p.m. (Figure 6). In the evening, people observe more birds than expected, and in the morning, less birds than expected compared with the prediction formed by calculating the product of bird vocalization activity (based on PAM species observations) and user recording activity. Application recordings also contain more bird observations than PAM recordings (Figure 6).

Figure 6

User behaviour and reliability of user-based annotations

With a great number of users comes a great number of unique recording patterns. The species accumulation curves of active users show that while some users seem to focus their efforts on maximising new species entries into their lists and obtain a new species on almost every third recording, others seem to be more interested in producing a large collection of recordings from their surroundings without actively targeting new species (Figure 7). However, all users seem to be motivated to record new species, and even the most slowly increasing species accumulation curves of users exceed those of PAM recorders. Unsurprisingly, most users made opportunistic recordings (i.e., to “hunt” for more species), which were most likely targeted at one or more vocalizing birds, whereas a large proportion of PAM recordings contains silent periods.

Figure 7

To evaluate how reliable the citizen scientist–annotated bird-sound data is, we rigorously validated the bird application identifications as a proof-of-concept. The results show that there is marked variation in the reliability of users (Figure 8). In our sample of 50 users, the average user-specific correct answer percentage was 73%, while the lowest percentage was 46% and highest 98%. Out of the 2,500 user-validated cases, the majority were true negatives (38.2%), followed by true positives (35.7%), false positives (17.6%), and false negatives (8.5%).

Figure 8

How people welcomed the application

From participants’ point of view, mobile applications can be very engaging and are accessible to many; from researchers’ point of view, the application interface can be harnessed to support interoperability across platforms and to address many research questions. Our initiative sparked significant public engagement as reported in other previous citizen science studies (Haklay 2013; Lehikoinen et al. 2023; MacLeod and Scott 2021; Santangeli et al. 2023; Sullivan et al. 2014). The bird-sound application attracted more than 140,000 users (ca. 2.5% of the Finnish population of 5.5 million) during a single season through which they made close to three million recordings from which almost six million birds were identified by an AI-powered classification algorithm (H1; see Table 1). For the same period (April 1^st–July 30^th, 2023), the bird observation database of the BirdLife Finland (tiira.fi) contained 859,687 observations (retrieved December, 4^th 2023). These observations were of significantly more individuals (ca. 241 million) because numbers are often greater when large flocks can be visualized. Thus, when effectively advertised, citizen scientists can become engaged in large-scale scientific research and accumulate data with remarkable intensity (Frigerio et al. 2021; Haklay 2013; Land-Zandstra, Agnello, and Gültekin 2021; Sullivan et al. 2014). The role of media and effective communication is crucial for the success of citizen science; high visibility attracts more people, which translates to more data. However, to provide scientifically robust data, the instructions must be as clear as possible to make the task accessible to the public.

Spatio-temporal distribution of recordings

The spatial sampling of user observations was widespread. However, recordings were focused close to where people normally go outdoors (H2), which causes sampling to be correlated with higher human population densities (Amano, Lamming, and Sutherland 2016; Balázs et al. 2021; Tiago et al. 2017a). Most observations were recorded within 10 km of the centre point of all recordings made by the user. Nevertheless, these recordings still accumulate a network of data points and thus serve as significant contribution in those locations. A better design for the systematic point count approach and expansion of the point count network into less populated areas would be important in mitigating potential sampling bias. As it is, the application encouraged people to record a high number of species by granting “awards” when a user’s species count reached certain levels. Gamification could be one solution to mitigate sampling biases by rewarding users recording data from underexplored areas across the seasons. Since regular and both spatially and temporally dense interval recordings are the most useful data for scientific research of birds, the application should guide and encourage people towards producing more standardised (e.g., in terms of location, time, and duration) natural sciences data (Dickinson, Zuckerberg, and Bonter 2010; Frigerio et al. 2021; Lehikoinen et al. 2023). We also note that of the bird fauna of Finland, many of the species occur elsewhere in the Europe. Thus, expanding sampling to cover the entire continent could be achievable in terms of existing training libraries of bird sounds, should the logistical requirements of data storage be addressed first.

The temporal sampling of recordings followed the progression of summer and the daily activity of birds (Moran et al. 2019; Robbins 1981). The recording activity over the season was highest in early summer (H3), April through May, after which it levelled out with the progression of summer in June, and it diminished in July. Throughout the summer, the recording activity was higher on weekends than weekdays (H4). Similar behaviour has also been observed in other citizen science projects (Di Cecco et al. 2021; Courter et al. 2013). The seasonality sets a challenge for the application. On the one hand, the application can identify the species, for example, from flight calls outside the breeding period (when the birds are less vocal), but on the other hand, it may require extra effort to motivate users to observe the sounds during quieter seasons (Thompson et al. 2023). The data from following seasons will show if the decrease in recording activity was because of the lower vocalization activity of birds towards the end of summer or because of user fatigue. Also, as some people use the application to learn to recognize bird species, it might be that those users cease to use the application when they can identify common species without it. The temporal recording behaviour of the users within the course of a day did not match the daily activity peaks of the bird singing activity (H5). The bird singing activity peaked at sunrise at approximately 5 a.m., but recordings peaked later in the morning at approximately 10 a.m. A more formal time-instructed sampling design (i.e., less opportunistic for gaining new species based on user behaviour and more focused on long-term recording) would help to overcome the challenges in temporal as well as spatial sampling (Frigerio et al. 2021; Knape et al. 2022; Tiago et al. 2017a).

User behaviour and reliability of user-based annotations

The recording behaviour of the users varied markedly. Some observers focussed their efforts on maximising new species entries into their lists, and they can be characterised as species collectors. At the other end of the scale are local observers, who are very committed to producing a high number of recordings in their own locality. These users actively make new recordings even if there are not always new species available, which makes the sampling temporally more balanced. As our user data is anonymized, we cannot evaluate potential demographic difference among user groups (Tiago et al. 2017b). However, regular sampling executed by a high number of observers provides valuable data about the occurrence of birds (Sullivan et al. 2014). These recordings could thus be used to predict progression of migration, especially when the recordings are made through the congruent interval sampling scheme, and the efforts of species collectors advance data collection on infrequent or rarely heard species.

Overall, our results highlight that citizen science–driven recording efforts complement sampling through PAM. In passive monitoring, the sampling effort is easy to balance temporally, and data of both presence and absence of birds is produced without bias caused by targeting opportunistic recordings to certain bird individuals or species. However, the sampling network comprising thousands of citizen scientists offers spatial density and coverage far beyond passive monitoring scenarios. Recordings made by users also contain, on average, more birds than passive recordings, which might be useful, for example, when collecting observations of rare species.

In this work, the accuracy of user annotations varied notably among the predicted categories as follows (H6): True positives (36.7%) represented the ideal way the application should yield data. True negatives were the most common category in our annotated cases (38.2%). These data are useful for improving the classification performance of the AI model. The obvious reason for true negatives are the presence of species that are vocally similar to the suggested species (such as common versus tree sparrow or common versus arctic tern); however, these problems may be tackled by extensive training sets to mitigate the similarity challenge. As the AI-powered classifier extracts a great number of parameters from a single vocalisation, being able to separate difficult species pairs is dependent on the extent of the training material. True negatives also included low-frequency anthropogenic sounds such as human speech and mobile phone or machinery noise, which automatic identification models tend to confuse with low-frequency bird vocalizations. False positives (17.6%) could be due to human misidentifications, notably a large number of users want to learn more about bird sounds but are not yet experts at identifying birds. It is also possible that the species was present in the original recording, but the highest model prediction pointed at wrong part of the recording and the target species was thus not detectable in the 4 s sample. Another user-based error could be that the bird was seen, but not heard, and got annotated based on wrong evidence. Also, purposefully wrongly annotated cases occurred (e.g., sometimes the recording was evidently not from a bird, but still annotated as a bird sound). False negatives (8.5%) could emerge, for example, from birds that were flying over but got missed (such as high-pitched siskins). Also, some users have reported annotating certain identifications as false as a means of trying to teach the AI model about another more prominent bird voice in the soundscape.

Ensuring data quality is often considered a potential weakness of citizen science studies (e.g., Balázs et al. 2021). For example, due to the varied quality of the annotations, to obtain a reliable data set, the observations identified by users may require manual filtering and validation. Based on our results, we do not recommend that user-identified data is utilized without any validation procedure. However, the massive number of recordings enables targeted search of those that are valuable, and those can then be manually validated with reasonable effort. Our results encourage the use of automatic species identification models in the collection of species occurrence data. To circumvent data quality problems that characterize many citizen science projects (i.e., due to the variable identification skills among citizen scientists), our approach stores the raw audio in a centralized repository, enabling rigorous validation and re-analysis in the future as the identification algorithm gets improved. This enables analysing past data in the near future, and allows for the potential to observe changes in distribution and timing of migrations, which is not within the reach of other common sound-classification applications. After the first year, the identification model has been re-trained using some of the data collected during summer 2023. The species list of the new model has been extended to cover 263 species (which covers all species that are regular annual visitors in Finland), and model accuracy and calibration have been improved. We caution, however, that many species that are naturally “silent,” such as many shore- and waterbirds, are beyond the reach of detection by sound through our application. Alternative observation methods may be required to detect these species’ presence.