From definitions to implementation – A guide to collect and apply the Lancet Commission on Global Surgery indicators: An Utstein consensus report

Perioperative care is a critical component of resilient healthcare systems. In large part, this is due to the overwhelming burden of diseases that require surgical, obstetric, and anaesthesia management, including but not limited to: trauma and emergency care, communicable diseases, noncommunicable diseases, maternal care, and congenital anomalies (Bickler et al., 2015; Rose et al., 2015; Shrime et al., 2015). Due to their interdisciplinary nature, perioperative interventions are embedded within the Universal Healthcare Compendium of essential services and play a key role in achieving the Sustainable Development Goals (Roa et al., 2019; Rose et al., 2021; World Health Organization, n.d.). More recently, perioperative interventions and resources – including a skilled workforce, oxygen, personal protective equipment, and mechanical ventilators – were crucial for managing surge conditions during the COVID-19 pandemic and will be crucial to address the backlog of surgical cases that resulted from delays during the pandemic (Rose et al., 2021; World Health Organization, 2023).

Ensuring quality care is accessible and delivered requires target-setting and regular monitoring. Unfortunately, most healthcare systems are data deserts regarding information on perioperative healthcare resources and delivery. Despite a growing global enthusiasm for health information technology, there is a paucity of digital infrastructure dedicated to data on surgical, obstetric, and anaesthesia care within Ministries of Health (DHIS2, n.d.; Juran et al., 2019). A recent review demonstrated that less than half of countries report their surgical volume and less than 10% report perioperative mortality (Holmer et al., 2019). Deficient data ecosystems likely mask disparities in access and quality within and across settings, with catastrophic repercussions for vulnerable populations in Low- and Middle-Income Countries (LMICs) (Biccard et al., 2018; Cossa et al., 2021).

In 2015, the World Health Assembly unanimously passed Resolution 68/15, calling for meaningful and reliable surgical and anaesthesia care measures following its designation as a component of universal health coverage (World Health Assembly, 2015). During the same year, the Lancet Commission on Global Surgery (LCoGS) introduced a suite of six metrics to inform national surgical plans and monitor scale-up toward 2030, namely: (1) timely access to essential surgery within two hours; (2) workforce densities of specialist surgeons, anaesthesiologists, and obstetricians; (3) volume of surgical procedures: (4) reporting of perioperative mortality rate; and (5 and 6) protection against impoverishing and catastrophic health expenditures (Meara et al., 2015). Since then, many countries have also ratified health care policies in the form of National Surgical, Obstetric, and Anesthesia Plans (NSOAPs) that integrate these metrics into their plans (United Nations Institute for Training and Research, 2020; Program in Global Surgery and Social Change, n.d.). The World Bank (2016) and World Health Organization (WHO) (2015) subsequently adopted the six metrics into their World Development Indicators and Global Reference List of 100 Core Health Indicators, which represent premier compilations of international statistics on global development. Roughly 70% of the world lives without access to safe and affordable surgical, obstetric, and anesthesia care (Alkire et al., 2015). As governments recognize this need and work to improve their health systems for surgical, obstetric and anesthesia care, they have an opportunity to create data ecosystems based on evidence for inclusion (or integration) into health policy (Gajewski et al., 2018; Perez-Iglesias et al., 2023; Truché et al., 2020).

Despite initial enthusiasm, three critical problems were encountered during the early adoption of the LCoGS metrics. First, inconsistent interpretation of data definitions significantly limited informed conversations around policy and advocacy (Davies et al., 2021). For example, the LCoGS definition of specialist workforce is restricted to medical doctors. Yet, the contributions of nonphysician healthcare providers are included in LCoGS metrics such as surgical volume, perioperative mortality, and access (Cossa et al., 2021). Second, sub-stratifications or adjustments may be necessary to ensure comparability. For example, perioperative mortality is known to vary between emergency and elective procedures, but without correcting for the proportion of emergency procedures, it is impossible to know whether discrepant perioperative mortalities reflect a true difference in quality of care or simply an underlying difference in case mix. This is especially problematic when comparing convenience samples from different settings where case mix cannot be assumed to be comparable. Third, the burden of data collection overwhelms the local capacity to maintain a data ecosystem, especially when guidelines are published in only a few languages. These barriers hindered the implementation of LCoGS metrics at the population level. Metrics of value need to be accessible, comparable, and utilizable (Davies et al., 2021; Moher et al., 2010; Van Panhuis et al., 2014).

The Utstein Consensus is a well-established process for bringing stakeholders together to collaborate on global concerns requiring consensus and action (Laerdal Foundation, n.d.; Otto et al., 2021). The Utstein formula aims to improve survival by leveraging advances in medical science, improving education efficiency, and facilitating local implementation. The consensus methodology uses a structured process to produce several outputs, including guidelines and protocols within the healthcare space. The process includes convening virtual and in-person workshops to refine inputs from experienced content experts and prior research to reach a final consensus and establish a plan for future work, follow-up, and dissemination (Otto et al., 2021). Given the processes and objectives inherent in these consensus guidelines, formal ethical approval was not pursued as this project involved no study subjects.

The first Utstein working group for global surgery, obstetric, and anaesthesia metrics met in 2019 to clarify and modify the Lancet Commission on Global Surgery indicators, where appropriate (Davies et al., 2021). This led to six metrics being reduced to five and creating basic, intermediate, and full indicator lists to collect these indicators progressively (Davies et al., 2021; Meara et al., 2015). The goal of the follow-up Utstein consensus meeting in 2022 was to discuss implementation strategies for data collection. The focus was to expand on the outputs from the previous meeting to clarify methodologies for data collection for the basic and intermediate datasets and produce indicator definitions, metadata, and data dictionaries. In doing so, this fulfills the aim of establishing metrics that are accessible, comparable, and utilizable (Laerdal Foundation, n.d.).

The steering committee assigned 40 globally representative delegates to one of the five working groups; each focused on a single global surgery indicator – geospatial access, workforce, volume, perioperative mortality rate (POMR), or financial risk protection from catastrophic expenditure (Davies et al., 2021). Delegates were broadly chosen from fields of surgery, anaesthesia, obstetrics, health policy, data science, and public health. Their country of residence is summarized in Table 1; representing 25 countries, 18 of which were LMICs. Each working group was assigned a lead and deputy to moderate the consensus discussion and that the Utstein format was followed. These groups met during a two-year process of virtual engagements before and after an in-person meeting in Norway. The outputs of these groups were preliminary data dictionaries and metadata templates based on the outputs from the first meeting in 2019. These initial documents provided an organized framework for each indicator to assess the methodologies required to gather the datasets at an inperson symposium in Norway.

Thirty delegates attended an in-person meeting from September 19-20, 2022 at the Utstein Abbey, Mosterøy Island, Norway. Attendees comprised international experts in surgery, anaesthesiology, obstetrics, health policy, and data science from 19 countries including 11 LMICs (Burundi, Gambia, Ghana, Honduras, India, Nepal, Nigeria, Mozambique, Rwanda, South Africa, and Uganda). Of the attendees, 40% were female. Anaesthesiologists accounted for 40% of the attendees, 30% were surgeons/obstetricians, and the remaining were other related professionals (technical officers, global health researchers, and representatives from the World Bank, World Health Organization, and the Demographic Health Information Software group). The format was similar to the first Utstein Global Surgery Metrics consensus meeting in 2019, with participants being assigned to one of five working groups specific to each indicator. Where possible, these working groups for the in-person meeting contained the lead or deputy from the pre-meeting working groups to ensure continuity. It was not feasible to bring all members of the virtual working groups to the meeting due to space limitations (Davies et al., 2021).

At the Utstein Abbey, individual data points developed by the pre-meeting working groups were considered by each relevant working group with attention to the operationalization and practicality of collecting the data. Once the data elements to be collected were agreed upon by majority vote, breakout sessions focused on detailing sampling strategies for gathering these data elements. Following agreement within each working group, the groups reconvened to present their recommendations to the entire cohort of attendees, thereby allowing for further questions to be addressed and a final consensus among the whole group to be achieved. All topics from Utstein were discussed secondarily in virtual format by the full workgroups after the meeting. Workgroups convened until consensus was reached on each indicator’s metadata and data dictionaries. Where disagreements occurred, additional discussion was undertaken, and finally the majority vote was favored. For example, delegates concerned with a succinct list of surgical procedures convened on multiple occasions to discuss available frameworks of surgical procedure listings and codify a list of recommended procedures for data collection.

Several concrete deliverables were created using the framework described above over two years of engagement. First, metadata sheets to provide context for data collection will guide researchers, administrators, and policymakers in implementing and interpreting data in perioperative healthcare. Second, data dictionaries specify the discrete data elements to be collected for each indicator, including data inputs and their sources. Together, these constitute a “how-to” guide for data collection. Third, a call for proposals to do two pilot studies to confirm the practical feasibility of collecting this data in “real-world” conditions and provide refinements where necessary.

The success of a how-to guide for data collection rests in the ease with which standardized tools translate into action. Each of the LCoGS metrics carries unique nuances for implementation. Metadata and data dictionaries help to address pragmatic challenges of comparability by considering the entire life cycle of data, including generation, collection, processing, storage, management, analysis, visualization, and interpretation (Harvard Business School Online, n.d.). Beyond data definitions, this broad view encourages the implementer to begin with the end in mind in developing a robust data ecosystem.

A standardized template for metadata (Table 2) was adapted from the one used for metadata for the United Nations’ Sustainable Development Goals (SDGs) (United Nations, 2015). Within the SDG framework, the metadata delineate technical details for collecting 230 indicators to report progress towards 169 targets in achieving 17 SDGs (United Nations, n.d.). The SDG metadata were designed to situate each SDG indicator within the broad framework of the SDGs. In doing so, each indicator’s metadata corresponds to a particular target under the umbrella of a particular SDG. Component domains of the metadata include organizations responsible for global monitoring, definitions and concepts, units of measure, classifications, data sources, pragmatics of data collection (i.e. method, calendar, compilers, and release), institutional mandate, limitations, methods of computation, validation, treatment of missing values, and sample calculations.

The LCoGS metrics are not intrinsically imbedded into the SDG framework, but this metadata template was selected due to overlapping themes and/or methodologies and to ease use by those countries already using the SDG framework. There is also overlap in themes; LCoGS indicators support themes from SDG 3 to “ensure healthy lives and promote well-being for all at all ages” (United Nations, 2015). Specifically, Target 3.8 aims to “achieve universal health coverage, including financial risk protection, and access to quality essential health-care services”. Various SDG indicators in Target 3.8 are strikingly similar to LCoGS indicators. These “related indicators” with their LCoGS counterparts include SDG Indicator 3.8.1 “coverage of essential health services” and SDG Indicator 3.b.3 “proportion of health facilities that have a core set of relevant essential medicines”, both of which parallel LCoGS indicator “geospatial access to surgical and anaesthesia care”. Similarly, SDG Indicator 3.1.1 “maternal mortality ratio” and SDG Indicator 4.6.1 “death rate due to road traffic injuries” parallel LCoGS indicator “postoperative mortality rate”. Likewise, SDG Indicator 3.8.2 “proportion of population with large household expenditures on health as a share of total household expenditure” parallels LCoGS indicator “percentage of the population at risk of catastrophic expenditure if they were to require a surgical procedure”. These commonalities supported the rationale for adopting the SDG template for organizing LCoGS metadata. Complete metadata for each indicator are available on the website for the World Federation of Societies of Anaesthesiologists (www.globalsurgerymetrics.org).

A discrete data dictionary was created to facilitate implementation across diverse data-gathering platforms.(Table 3) To prioritize implementation in LMICs – especially those engaging in data collection for the first time on a nationwide scale – the decision was made to focus on data dictionaries of the basic and intermediate datasets, as detailed previously (Davies et al., 2021). The exercise of creating dictionaries revealed key methodological considerations, such as the need to envisage the source of data and reconcile it with the level of computation. For example, the data inputs for indicators of workforce and volume of surgery are both derived from a healthcare facility’s physical working space. However, the basic dataset reports densities that are only reported at the national level, without identification of distinct facilities, and are thus referred to as “population-level” data elements in the basic package. This underscores that the workforce and surgical volume numerators must be coupled with a corresponding denominator (per 100,000 population) to avoid ecological fallacy. The intermediate dataset, which calls for substrata of data according to facility-level variables (i.e. type, level, and financing) and patient-level variables (i.e. age, sex, and American Society of Anesthesiologists (ASA) category), allows for sub-stratification of results for deeper analysis and interpretation of national trends.

Although the data dictionary does not represent a data collection instrument, its format should allow for rapidly translating standardized variables into existing data ecosystems. For example, a surgical procedure list was generated through the Utstein Consensus to harmonize data collection. It also allows for stratifying surgery volume and POMR by surgery types. Historically, surgical research has focused on narrow subspecialty domains, such as orthopedic surgery or trauma surgery. However, datasets limited to different types of patients obviate meaningful data comparisons in aggregate, making population-level estimates impossible. In 2015, the Disease Control Priorities (DCP) project (3^rd Edition) assembled a discrete package of surgical interventions for district hospitals. These procedures form the backbone of essential surgical interventions in the WHO’s Universal Healthcare Compendium (Disease Control Priorities Network, n.d.; World Health Organization, n.d.). Criticisms of the DCP3 package included lack of representation of various surgical subspecialties and, in some cases, an emphasis on clinical management that may or may not involve surgical procedures. Other attempts to define a standardized list of surgical procedures followed, including a global modified Delphi process involving 331 participants from 78 countries (Odland et al., 2021). However, these lists of procedures were incompatible with LCoGS indicators due to overlapping domains. The current data dictionary for the “intermediate” dataset was developed through Utstein consensus by combining elements of these and other procedure lists highlighted by participants.

The Utstein surgical procedure list includes 37 procedure types that can be implemented rapidly for population-level monitoring. (Table 4) The procedure list was developed using rigorous criteria through multiple rounds of consensus. First, the Utstein Surgical Metrics Group ensured surgical procedures were included from multiple surgical subspecialties, including obstetrics/gynecology, general/emergency/trauma, orthopedic, cardiothoracic, neurosurgery, plastic/reconstructive, ophthalmology, and dental/oral surgery. Second, all surgical procedures needed to be consistent with LCoGS indicators. For example, “laparotomy” as mentioned in LCoGS, is quite broad and may occur simultaneously with various other intra-abdominal procedures (splenectomy, colectomy, small bowel resection, fundoplication, etc). For the Utstein procedure list, all laparotomy procedure sub-types had to be collapsible into the LCoGS laparotomy category without redundancy, overlap, or missingness. Third, every effort was made for surgical procedures to reflect unique underlying capacity. For example, circumcision is listed in DCP3 but was excluded by the Utstein Surgical Metrics Group because it is often performed outside the operating room and because incidence of circumcision is sometimes linked to sociocultural norms instead of a pathologic diagnosis, making it unreliable as an indicator of performance. Adherence to this principle of reflecting unique capacity also meant that sometimes data granularity was compromised to eliminate duplicate capacity. For example, as a proxy for laparoscopic capability, laparoscopic cholecystectomy was included as distinct from open cholecystectomy, but laparoscopic appendectomy was not included as distinct from “appendectomy”. Another example is that internal and external fixation of long bone fractures was not differentiated between the upper and lower extremities despite these groupings carrying distinct epidemiology and anticipated perioperative outcomes. Extending this logic, it is important to note that all the 37 procedures listed in Table 3 can be further subdivided into more detailed clinical groups. For example, “drainage of dental abscess” includes the treatment of Ludwigs Angina, but is also more broadly inclusive of any dental abscess. Similarly, “colectomy” may include resection of the ascending colon, the transverse colon, descending colon, and potentially the terminal ileum or upper rectum; but the current list does not differentiate between these sub-classifications. However, the Utstein Surgical Metrics Group strictly adhered to the objective of curating as short a list as possible to ease the burden of data collection. As such, the procedure list presented here is not comprehensive and should be considered for population-level monitoring of health systems.

A major focus of our Utstein consensus process was to ensure that the proposed data elements from the 2019 Utstein working group can be feasibly gathered in diverse settings. From the beginning, care was taken to follow the goals of establishing metrics that are accessible, comparable, and utilizable. To confirm feasibility of standard data collection for the data elements agreed upon at the meeting, a call for proposals to test feasibility of data collection using the meeting’s outputs was issued after the meeting in Norway to confirm their utility or refine if necessary.

An independent committee of content experts reviewed eighteen proposals to ensure no conflicts of interest between the Utstein steering committee and the applicant pool. The judging criteria were based on impact, setting, comprehensiveness of indicators addressed, team, data collection/pilot delivery, feasibility, and cost-effectiveness of the proposed pilot study. Following this review, two pilots were launched in the spring of 2023 in Ghana and South Africa. The Ghana research team aims to assist the Ghana Ministry of Health in collecting all five LCoGS indicators according to Utstein modifications to inform Ghana’s National Surgical, Obstetric and Anaesthesia Plan (NSOAP) and to develop routine data collection of the basic set of indicators. The South Africa research team aims to demonstrate the feasibility of routine prospective data collection using the DHIS2 platform to assess POMR and surgical volume at the intermediate level, which includes risk-adjustment and indicated sub-strata (i.e. surgery type, severity, urgency, etc.). These pilots have concluded with anticipated publication in the near future.