Contextual Insights for Infectious Pathogen Surveillance Using the Epidemic Intelligence from Open Sources System

Use case selection

This evaluation retrospectively analysed outbreak‑related data from EIOS. Four use cases were selected through a literature review and consultation with the EIOS Core Team (ECT) at WHO. Each case represents a distinct geographic region and pathogen. Outbreak periods were defined using reference data from scientific or grey literature.

The first use case is an influenza outbreak in Brazil that started in the State of Rio de Janeiro due to the H3N2 Darwin type A influenza strain. The outbreak period began in November 2021 and ended in December 2021, with cases peaking in late November [12].

The second use case is a COVID‑19 surge in Kenya, driven by the introduction of the Omicron variant BA.1. Sources define this ‘fifth wave’ of COVID‑19 to be between mid‑November and mid‑December of 2021 [13, 14].

The third use case is a surge of measles in Afghanistan, one of many that have occurred in the past several years. While there were relatively low levels of transmission in 2019 and 2020, cases increased dramatically in 2021. Using a report from the WHO Eastern Mediterranean Regional Office as reference data, the outbreak period is defined from 1 January 2021 to 31 December 2022 [15].

The fourth use case pertains to a diphtheria outbreak in Nigeria. The outbreak period is based on reports from Nigerian Centre for Disease Control (NCDC), which documented a surge of suspected and confirmed diphtheria cases between 9 May 2022 and 1 October 2023 [16–18].

EIOS system boards

EIOS boards are customized search filters for monitoring diseases, symptoms, health threats, or contextual factors in a specific geographic area and time frame. Board results consist of web‑based information (e.g., press releases, news articles, social media posts) mined from open sources. System users have the option to use ‘pinning’ and ‘flagging’ features of the EIOS system to mark items of interest among these search results. Boards were created for each use case, filtering items by geographic region, disease, and combinations of multi‑lingual keyword searches known as categories. Time filters included two weeks before and after outbreak periods to ensure detection of important events and contextual insights.

Item triage

The items returned from each board were triaged, ‘which involves the sorting of data and information to discard duplicates and to exclude disinformation, irrelevant information, and false information to identify real events’ [19]. The triaging protocol was developed using a combination of documentation from various EIOS users’ published workflows and a standard operating procedure that was previously developed by the ECT for broad use. Non‑English text was translated using a web browser extension before review.

In a primary review, the analyst scanned items for contextual information using only article titles and summaries, which are automatically generated in the EIOS system using an abstractive summarization algorithm. The system’s flagging feature was used to mark and save items that appeared relevant to the outbreak use case. The analyst also took notes on common themes related to contextual insights that were observed in the summaries.

A secondary review involved reading the full text of the flagged items. These items were read starting with those imported into the EIOS system first (oldest import date). Items were saved using the pinning feature if they were relevant to the outbreak and contained new information that was not previously captured in previously pinned item. The analyst also noted thematic patterns in the contextual data among pinned items.

Qualitative coding

Observation notes from triage informed a qualitative codebook for each use case. Pinned items were coded using an iterative process, refining themes through repeated analysis and comparison. Each outbreak use case had a distinct list of thematic codes, with some common across cases [20]. This coding scheme resulted in distinct lists of thematic codes for each outbreak use case, although some themes could be common across all use cases. Supplementary File 1 contains the final code lists and assignment criteria.

Two primary types of codes were assigned to EIOS system items: public health events and contextual data. Event data included cases, suspected cases, deaths, and hospitalizations. Contextual data focused on those factors that impacted or informed outbreak response. It also included risk reduction factors such as public health and social measures (PHSM) and MCMs.

A single item could be coded more than once across the full text. For example, a single item might provide new event data for many states within a country in addition to sharing new information about local healthcare capacity and other contextual themes. Therefore, the cumulative frequency of codes may exceed the number of items.

Finally, contextual data codes were quantified by epidemiological week, using the EIOS import date, to track thematic trends during outbreaks. Qualitative descriptions of salient contextual themes also accompany the quantitative data.

Outbreak Use Case 1: Influenza type A in Brazil (2021)

The board representing Use Case 1 resulted in 11,982 articles imported in EIOS between November and December in 2021. Search criteria included all influenza‑related categories and articles that mentioned any geographic region within Brazil. After triage, there were 1392 articles (11.6%) remaining that contained unique, outbreak‑relevant information. Among these articles, there were 751 outbreak event‑related codes and 861 codes applied to contextual insights (Table 1).

The contextual data codes provide information about environmental, sociopolitical, and other themes at various stages of the H3N2 outbreak in Brazil (Figure 1). Public health infrastructure issues made up nearly one‑quarter of the codes, primarily related to healthcare capacity, including bed shortages and doctor shortages. Items covering data availability and accessibility observably increased as the influenza cases spiked; these items were local articles, press releases, and other updates on a hack that affected the national Ministry of Health and its data reporting systems [19]. Shortly after the initial confirmation of the hack, additional reports were released to explain the extent of the data infrastructure problem and the kinds of influenza surveillance barriers it posed as the Ministry of Health attempted to recover from the data breach [21, 22].

Figure 1

Other important data obtained from the EIOS system during this outbreak related to clinical presentation, notably the differences in disease severity between influenza variants. In addition, mass gathering warnings and cluster detection codes were applied before, during, and after large gatherings, such as Carnival. Officials in various Brazilian localities would release warnings and social distancing guidelines prior to such events, while media would report clusters of confirmed influenza cases after the gathering.

Health behaviours, knowledge, or attitudes were related to resistance to precautions to prevent transmission of both COVID‑19 and influenza. Some articles noted resistance among political authorities, resulting in relaxed precautions. Articles coded with this theme appeared at the very early stages of the influenza outbreak, as well as the later epidemiological weeks when cases drastically decreased. Vulnerable groups were also often identified, particularly among healthcare workers and teachers. This code was also applied alongside the environmental conditions code, which most often referred to disease vulnerability concerns in the aftermath of extreme flooding or heat in some areas.

There were relatively few PHSMs reported during the outbreak compared to MCMs, which mainly included vaccination campaigns and increasing hospital capacity. PHSMs were mainly cancellations of mass gatherings and masking mandates. These became more frequent during Epidemiological Week 14, which coincided with the peak of the outbreak according to the reference data.

Outbreak Use Case 2: COVID‑19 Omicron variant in Kenya (2021)

The board representing Use Case 2 resulted in 7314 articles in EIOS between November and December in 2021. Search criteria included all categories containing keywords related to COVID‑19, SARS‑CoV‑2, and its variants and articles that mentioned any geographic region within Kenya. After completing the pre‑defined triage process and qualitative coding, 427 articles (17.1%) containing unique, outbreak‑relevant information remained. After performing thematic coding on these triaged articles, there were a total of 343 event‑related codes. There were 165 codes applied to contextual insights, which are summarized and categorized in Table 2.

Figure 2 exhibits the contextual data coded across EIOS system items over the outbreak period. Public health infrastructure was a salient theme in this use case, mainly covering healthcare capacity issues and vaccine shortages. Periodic reports mentioned reduced lab testing capacity, accounting for the unavailability of variant information.

Figure 2

Another important contextual theme was health behaviour, attitudes, and knowledge. The ‘fifth wave’ of COVID‑19 in Kenya emerged as the country relaxed restrictions, leading to a reinstatement of certain measures such as curfews. Items reported resistance to continued curfews and a proposed vaccine mandate. Other articles mentioned vaccine hesitancy in faith‑based communities and increased travel post‑lockdown. Some articles in this theme included speculation about the impact of media disinformation on vaccine uptake.

Articles coded as mass gathering warnings focused on reported increases in hotel bookings and renewed tourism influxes. Vulnerable groups were also reported, particularly healthcare workers, truck drivers, school‑aged children, and the rural poor. Clinical presentation‑coded items were less frequent in this use case because COVID‑19 symptoms and virological mechanisms were already common knowledge among public health professionals. However, speculation about the high transmissibility of Omicron based on patient observations in Kenya was coded. These codes also applied to unique outcomes and indications of COVID‑19 disease severity if they were observed in Kenyan health facilities during the fifth wave.

Most mitigation measures detected were county and province‑level vaccine campaigns, including vaccine donations from foreign agencies. The results also showed an increase in PHSMs reported after Epidemiological Week 8, corresponding with the peak case volume indicated by the WHO reference data. PHSMs included increased border screenings, testing mandates, and travel restrictions.

Outbreak Use Case 3: Measles in Afghanistan (2021–2022)

The board representing Use Case 3 resulted in 1745 total items from 2021 January to 2022 December. Search criteria included all measles‑related categories and articles that mentioned any geographic region within Afghanistan. After 2 rounds of reviewing, coding, de‑duplicating, 109 articles (6.2%) containing unique, outbreak‑relevant information remained. After performing thematic coding on these triaged articles, there were a total of 128 codes applied to text that provided event data. There were 71 codes applied to contextual insights, which are summarized and categorized in Table 3.

Figure 3 illustrates the contextual data obtained from the EIOS system during the outbreak period. Key public health infrastructure context coded among the EIOS system items was related to barriers to providing care and vaccination in remote areas and medical personnel shortages. The potential issue of vaccine resistance was raised early in the outbreak because of distrust of vaccination campaigns in the region [23]. Although most of the symptoms reported were based on previously known clinical evidence, items were coded (clinical presentation) when measles severity was linked to local conditions. For example, local health officials stated observations that symptoms worsened among children experiencing malnutrition. In addition, earthquakes were coded as environmental conditions that impacted certain provinces’ ability to receive care and therefore increased their risk of vulnerability to many diseases, measles included. The main contextual insights that preceded the peak of the outbreak were about unvaccinated populations and disease risk among socially vulnerable groups, including children, pregnant women, and those living in remote or inaccessible villages.

Figure 3

MCMs in various regions of Afghanistan were reported throughout the outbreak. PHSMs reported in media increased about halfway through the outbreak during Epidemiological Week 20. Frequently mentioned mitigation measures included vaccination campaigns, donations and volunteers increasing healthcare service capacity, and expanding surveillance in remote areas.

Outbreak Use Case 4: Diphtheria in Nigeria (2022–2023)

The board representing Use Case 4 resulted in 1309 articles from 2022 May to 2023 October. Search criteria included all diphtheria‑related categories and articles that mentioned any geographic region within Nigeria. After completing the pre‑defined triage process and coding the remaining articles, 158 articles (8.3%) remained. After performing thematic coding on these triaged articles, there were a total of 406 codes applied to text that provided event data. There were 129 codes applied to contextual insights, which are summarized and categorized in Table 4.

The most salient contextual themes included low vaccine uptake and barriers to providing care in remote areas (Figure 4). Articles coded with these themes described low vaccine turnout, though only a minority attributed this to health behaviours, knowledge, and attitudes such as vaccine hesitancy. Rather, reporters wrote about the inaccessibility of vaccines in remote locations. Economic challenges leading to reduced testing capacity and doctor shortages were a major theme related to public health infrastructure. This also included vaccine and medicine shortages, but these supply challenges were less common. Ongoing lab testing capacity was reported as a cause of data availability issues throughout the outbreak period.

Figure 4

Most articles provided information about known symptoms of diphtheria as a warning for parents of children and providers; however, some media reports specifically outlined the clinical presentation of patients observed in Nigerian facilities, and these were coded when a new symptom or description of disease severity was provided. Vulnerable groups were also coded the first time a group in Nigeria was reported as being at‑risk; these included the unvaccinated, healthcare workers, and asthmatic individuals.

Most diphtheria mitigation efforts during this outbreak were pharmacological (vaccination and treatment distributions) and conducted at the national level. PHSMs were also reported throughout the outbreak, but these items in the EIOS system increased in Epidemiological Week 38. This corresponds to approximately five weeks after the peak of the outbreak, based on the reference data. A common form of preventive measure reported was community education to encourage vaccination.

Limitations

It is important to note that the data resulting from each EIOS system search result is based on the web‑based sources being mined. While sources in the EIOS system are continuously being updated and added, more recent use cases may have additional web‑based sources to inform outbreak response and detection. Therefore, the retrospective analysis is not necessarily reflective of the current system. It is also important to note that there may be discrepancies between the EIOS item import date and time and the time at which the item was published to the web; however, previous EIOS system evaluations found generally timeliness of event detection [24–26].

In addition, there are fewer sources in the EIOS system from certain states, as exhibited in the Afghanistan use case, resulting in fewer triaged items and limiting the generalizability of the results. This presents an opportunity for additional collaboration with these states on source selection and integration into the EIOS system.

The usefulness of contextual data may be subjective for ministries of health and other authorities, depending on the specific themes that are found. Some themes may reflect well‑known issues, for instance, that do not need an EBS tool to uncover in an outbreak scenario. Finally, while manual thematic coding carries the advantage of flexibility and accuracy when compared to most automated technologies available, incorporating the methods employed in this study in routine surveillance would require substantial human resources.

Public health impact

These qualitative data can be potential warnings for increased disease impact in the early stages of an outbreak. In addition, thematic trends and patterns across outbreak periods might help identify future risks and areas of need for improving outbreak response. These example results illustrate how analysis of text‑based PHI system data can help inform authorities on thematic patterns in public health discourse to potentially shape communication priorities and campaigns as outbreaks progress.

This evaluation may serve as a basis for manual or AI‑driven qualitative coding practices in routine disease surveillance. EIOS system users who monitor diseases at the national and local levels may predetermine themes of importance based on these results and other existing evidence about contextual factors that affect disease spread, outbreak response, and detection. The findings can also support broader, more comprehensive pre‑built categories in the EIOS system, which currently tend to include keywords that are limited to specific diseases and pathogens. Examples can include local attitudes, geopolitical concerns, or environmental conditions that may increase or mitigate disease risk. Health officials may establish a system for maintaining items that are manually categorized with contextual themes of interest, so that thematic codes may continue to be refined with different users’ interpretations. Improving on best practices for monitoring pathogens using reflexive coding schemes may highlight how to use contextual insights to parse risks across the population and potentially distinguish contextual patterns surrounding emerging diseases, seasonal spikes, and off‑season surges.

Future research efforts to improve upon the qualitative approach of this evaluation include exploring advanced methods, such as topic modelling or network analyses, to understand the relationships between contextual factors, case surges, and other public health events. This may be triangulated with additional ‘gold standard’ indicator data or qualitative information from users in the EIOS system’s community of practice. These methods of ‘contextualised intelligence’, among other qualitative methods, may not only strengthen PHI but also improve international and cross‑sector collaborations [27]. Therefore, the inclusion of contextual insights in future evaluations and standard operating procedures has the potential enhance the predictive power and decision‑making capacity of these event‑based, collaborative surveillance systems through collaborative and interdisciplinary health surveillance.