The difference in clinical knowledge between staff employed at faith-based and public facilities in Malawi

Introduction

A major problem for poor people in low-and middle-income countries (LMICs) is access to quality healthcare.¹ Theoretical and empirical research, predominantly from high-income countries, shows that the management of healthcare organisations determines how healthcare is delivered which may impact quality of care.^2,3 LMICs are characterised by a higher proportion of healthcare delivered by non-governmental providers which may result in variations in quality of care depending on health facility ownership. Given the low density of healthcare providers in LMICs, access to differently owned facilities may contribute to unequal access to quality healthcare.

In Sub-Saharan Africa (SSA), non-profit faith-based providers (FBPs) are estimated to deliver 30% to 50% of all healthcare.⁴ FBPs are frequently perceived by patients and governments to provide higher quality of care compared to public providers often due to anecdotal evidence.⁵ The historically rooted level of trust has motivated many public-private partnerships across SSA and has reduced the perceived need for contractual accountability mechanisms.^6,7 However, large-scale evidence regarding quality differences across ownership which accounts for confounding variation is limited. The measurement of healthcare quality is complex and multidimensional and has traditionally been divided into structural-, process-, and health-outcome quality.¹¹ Given data limitations across many LMICs, most of the research investigating the differences in service delivery by religious non-profit and government providers focus on client satisfaction. Patients visiting FBPs report comparatively higher satisfaction compared to patients visiting both public and private for-profit providers.^{5, 12, 13} However, patient satisfaction may be impacted by case-mix variation and response or recall bias. Client satisfaction may also better reflect patients’ perceptions of respect and courtesy than the facility’s adherence to medical protocols.¹⁴

It has been shown that poor clinical knowledge is a greater constraint across African countries than other components of the healthcare system, such as drug availability or health worker absenteeism.⁸ Clinical knowledge is a key measure of the quality of treatment as it indicates the highest level of quality of care the provider would be able to deliver assuming access to relevant medical infrastructure and adequate incentive mechanisms. Additionally, competence has been shown to be highly correlated with effort.^{8, 9, 10} A previous study from Tanzania did not find a difference in a clinical competence score based on vignette data for four clinical adult and child cases between healthcare staff working for facilities managed by a church-organisation, compared to public or private-for-profit management.¹⁶ Otherwise, the majority of extant studies do not differentiate between FBPs and other private providers when analysing ownership differences using vignette data.¹⁷

We add to the existing knowledge on the relationship between faith-based, compared to public management of health facilities and clinical competence in LMICS, by investigating differences in clinicians’ knowledge of diagnosing and treating eight adult and child cases using novel and nationally representative data of the Malawian healthcare system. Approximately half of all facilities in Malawi are owned by the Government of Malawi. The Christian Health Association of Malawi (CHAM), an umbrella organisation of Christian (Catholic and Protestant) non-profit providers, manage 14.9% of the country’s facilities¹⁸ and is estimated to serve around 37% of the total Malawian population and 75% of the population in rural and remote areas.¹⁹ As such, the faith-based sector in Malawi plays an important role in providing healthcare for poor and vulnerable populations.

Data

The Malawi HHFA was conducted between November 2018 to March 2019 and is a census of all existing health facilities during the survey period. Data access was provided by Dr. Gerald Manthalu, Deputy Director of Planning and Policy Development, Ministry of Health, Government of Malawi. The HHFA includes detailed information on 1106 health facilities of which 101 are hospitals, 492 are health centres, and 513 are dispensaries, clinics, and health posts. Of these, 575 are public facilities and 531 private facilities, including 165 CHAM and 53 non-governmental organisations (NGOs). The survey collects comprehensive information about facilities, healthcare workers, and patients using different survey modules. Information on service availability and readiness is retrieved from the facility inventory, facility management and finance questionnaire, and the health worker roster. Health worker presence and absence is measured through unannounced facility visits. Client consultation experiences and characteristics are captured by client exit interviews. Health worker capacity to diagnose and treat common illnesses is assessed during a clinical vignette module.

Our analysis focuses on the clinical vignette data module. For this module, health workers were randomly sampled at each facility and a maximum of 10 health workers were selected per facility. In facilities where the number of health service providers was less than ten, all health workers who provided patient consultations were included in the sample. The HHFA was comprised of vignette data for 1,433 individual health workers, including 286 based at CHAM and 765 based at public facilities. We excluded 382 observations from facilities managed by other organisations that resulted in an analytical sample of 1051 healthcare worker observations.

During the vignette, the interviewer acted as a “patient” and provided a brief description of symptoms of a hypothetical case. Subsequently, the healthcare worker was asked to proceed exactly as they would under normal circumstances, asking questions about the history of the illness and performing necessary examinations and tests to which the “patient” provided predetermined answers. The questions, examinations, and proposed tests were then compared to a protocol checklist.⁸ Vignettes have been shown to be a valid tool for measuring clinical knowledge of health workers and have been widely used in research and medical training on account of being inexpensive and easy to use for diverse settings. Moreover, they also remove any concern about case-mix variation.^{8, 15}

The Malawi HHFA provided vignette information for eight common childhood, adult, and pregnancy related cases which constitute major global health challenges.⁸ Each case and the criteria used to define correct diagnosis and treatment (or management) are presented in Table 1 and 2 below. Table 2 shows the criteria used to define the correct diagnosis and management of the two maternal and neonatal emergency cases -postpartum haemorrhage and neonatal asphyxia. For management of these cases, all components of the criteria in Table 2 had to be mentioned by the clinician to be considered as correct.¹⁸

Methods

Our empirical model of the relationship between health facility ownership and healthcare worker competence and can be represented through the following estimation equation:

C_hfdrepresents the outcome variable denoting competence diagnosing, treating, or managing any case for healthcare worker h, in facility f, and district d. On the right-hand side, CHAM identifies the binary independent variable of interest that takes value 1 if a healthcare worker h is employed at a CHAM facility and 0 if they work at a public facility. Thus, the β₁ coefficient represents the change in the outcome variable C when the key independent variable changes from 0 to 1, i.e., for a CHAM healthcare worker compared to a government healthcare worker.

We follow previous studies^{14, 16, 17} and include a vector of control variables, denoted Cov, for cadre of healthcare professional, work experience, and facility type. For example, previous evidence show that higher cadres have higher competence and that work experience could have a negative effect on competence.^{14, 17} δ represents the inclusion of geographical covariates for Malawi’s five zones or 29 districts, keeping in mind the potential loss in power given the limited sample size. ε_hfdis a random error term.

All outcome variables, but the management of emergency maternal and neonatal cases, are dichotomous denoting correct or incorrect diagnosis and treatment. Therefore, for these cases, C in Equation 1 is a binary variable taking value 1 if a healthcare worker provides the correct diagnosis or treatment and 0 otherwise. Probit regression is applied to modelling binary outcome variables; therefore, we use a probit regression model when estimating the relationship between faith-based facility ownership and binary competence outcome variables. The predicted probability that a clinician reports the correct diagnosis or treatment for these cases is a function of an unknown continuous variable, and the probit regression models the relationship of this variable to the independent variables.

Correct management of post-partum haemorrhage and neonatal asphyxia (see Table 2) requires a higher number of actions; therefore, these outcome variables are provided on a scale 0-100. Given the continuous nature of these competence measures, we use Ordinary Least Squares (OLS) to estimate the relationship between facility ownership and the proportion of appropriate post-partum haemorrhage and neonatal asphyxia management actions. All statistical analyses were conducted using Stata v.^17.

Results

Tables 3 and 4 present the summary statistics for all outcome and control variables for the pooled sample in the first column to the left and for the sub-samples of staff working in CHAM and public facilities in the following columns, respectively.

In Table 3, we observed a higher percentage of medical doctors employed at CHAM facilities (5%) compared to public facilities (1%). The majority (57%) of all interviewed health workers at public facilities were medical assistants, but the share of this cadre was lower at CHAM facilities (38%). The difference was made up of a higher percentage of nurses and clinical officers at CHAM facilities (31% and 26% respectively) compared to government facilities (22% and 20% respectively). Concerning type of facility, the sampled CHAM clinicians were more likely to be working in a hospital (55%) compared to those in public facilities (29%) and fewer CHAM employees were found in health centres (42%) compared to government healthcare staff (62%). It is evident that CHAM caters to predominantly rural areas as 92% of all CHAM professionals worked in rural facilities; the corresponding percentage for government staff was 69%.

Turning to the competence measures, we noted a varied knowledge level of correct diagnosis across cases. The majority of healthcare workers were able to correctly diagnose most cases except common childhood illnesses with a co-morbidity. Only 37% and 38% of healthcare professionals correctly diagnosed diarrhoea with severe dehydration and malaria with anaemia, respectively. Moreover, clinical staff employed at faith-based and public facilities in Malawi displayed a higher average clinical knowledge per case than health workers from a sample of 10 sub-Saharan African countries for the same cases.⁸

Only 32% of the CHAM health workers could correctly diagnose diarrhoea with severe dehydration during a case simulation, which is relatively lower than the corresponding share of 39% of government health workers. On the other hand, the descriptive statistics show that correct diabetes diagnosis was higher for CHAM staff – 93% compared to 85% for government employees. Additionally, correct diagnosis of malaria with anaemia was five percentage points higher for government employees (39%) compared to CHAM employees (34%). For other cases, the differences in correct diagnosis by ownership were not larger than a couple of percentage points. CHAM staff also had a lower likelihood of correctly treating diarrhoea with severe dehydration (52%) compared to government staff (56%) but they showed a higher clinical knowledge of correct treatment of diabetes (85%) compared to government employees (75%) and correct diagnosis and treatment of diabetes: 84% vs 74%. Faith-based healthcare workers were also more likely to display correct knowledge of treatment of anaemia in pregnancy (41%) compared to government staff (36%).

Table 4 presents the descriptive statistics for the continuous variables. We observed that CHAM employees had on average worked 7.34 years in their profession, which is slightly longer than the 6.82 years on average for staff working in public facilities. Healthcare workers employed in public facilities work approximately five more hours per week (60.11) on average compared to CHAM health staff (55.03). The heterogeneity across ownership motivates the inclusion of the discussed facility and health worker characteristics as covariates in our regression models. On average, clinical staff report 52.82% and 54.5% of correct management actions for postpartum haemorrhage and neonatal asphyxia, respectively. The difference in competency managing emergency maternal and neonatal across ownership is marginal although CHAM staff on average report a slightly higher proportion of correct management actions.

Tables 5 – 9 report the Probit regression results for the association between faith-based ownership and the probability of correct diagnosis and treatment by case, as specified by Equation 1. The tables report average marginal effects which helps us to easily interpret the coefficient for CHAM as, on average, a percentage point change in competence for a CHAM healthcare worker relative to a comparable government healthcare worker. For all outcomes, we first display results for the bivariate model without any covariates and stepwise add sets of control variables for i) health worker characteristics, ii) facility type, iii) rural location and zone and lastly, iv) we substitute the zone for district fixed effects.

Table 5 presents the regression results for diagnostic competence of common child cases. The bivariate Probit model in column 1 shows that working for CHAM is associated with a 7.2 percentage point reduction in the likelihood of correct diagnosis of diarrhoea with severe dehydration compared to working at a public facility. The magnitude of the coefficient increases slightly to 9.9 and 8.1 percentage points, respectively, when the full set of controls are added in columns 3 and 4. No association is observed between ownership and correct diagnosis of pneumonia across the specifications in column 5-8. We note that observations are dropped when district fixed effects are included in column 8 due to success being perfectly predicted within some districts. For the case of malaria with anaemia, we observe a negative average marginal effect of 0.056 for CHAM health workers in the bivariate model in column 9. However, the coefficient becomes statistically insignificant and reduces in magnitude as controls are added.

Table 6 displays the regression results for the probability of correct diagnosis of adult cases. We observed a positive relationship between faith-based ownership and correct diabetes diagnosis in columns 1-4. The fully saturated model in column 4 indicates that CHAM health workers were on average 7.2 percentage points more likely to diagnose diabetes correctly than comparable public health facility staff.

The estimations for correct diagnosis of tuberculosis in adults and anaemia in pregnant woman (columns 5-12) do not reveal a statistically significant difference between CHAM and government clinical staff across any specification. Neither, do we find that ownership was related to variations in correct diagnosis of post-partum haemorrhage and neonatal asphyxia, see Table 7.

We note a negative association between CHAM ownership and knowledge of correct treatment of a child presenting diarrhoea with severe dehydration across specifications in columns 1-4 in Table 8. However, the coefficient of interest is only statistically significant in the fully saturated model which controls for zone, see column 3. The output from this specification shows, on average, CHAM clinical staff are 8.8 percentage points less likely to correctly treat the case, compared to similar government staff. When controlling for district fixed effects rather than zone, the estimate on being a CHAM employee reduces in magnitude and does not reach statistical significance, potentially, due to the loss of power. We do not observe any differences in treatment knowledge by ownership for the remaining child cases presented in columns 5-12.

As with the diagnosis of diabetes, CHAM clinical staff are also more likely to correctly treat diabetes, see columns 1-4 in Table 9. The last specification in column 4 that accounts for district variation shows on average, CHAM healthcare workers are 8.1 percentage points more likely to correctly diagnose an adult presenting with diabetes type 2, relative to a comparable government healthcare worker. Otherwise, in similarity with previous regression results for diagnostic competence of the remaining two adult cases, we do not observe any statistically significant associations between ownership and the probability of correct treatment of tuberculosis and anaemia during pregnancy. Likewise, the OLS regression results in Table 10 show a lack of a statistically significant relationship between faith-based ownership and the proportion of correct actions required to manage post-partum haemorrhage and neonatal asphyxia.

Discussion

This study has investigated the difference in clinical competence across staff employed at faith-based and public health facilities in Malawi. We analyse novel and nationally representative vignette data which evaluates health workers’ knowledge of the management of eight common medical cases during a hypothetical setting. Using regression analysis, we find that clinical staff at FBPs are less likely to correctly diagnose and treat diarrhoea with severe dehydration in children by 8.1 and 8.8 percentage points, respectively, compared to staff at public facilities. This is a rather non-negligible difference relative to the average competence rate as only 37% (55%) of the analytical sample of clinical staff can diagnose (treat) diarrhoea with severe dehydration correctly. On the contrary, CHAM healthcare workers are 7.2 and 8.1 percentage points more likely to correctly diagnose and treat diabetes type 2 in adults, relative to government healthcare workers. For reference, 87% (77%) of CHAM and government healthcare workers know how to correctly diagnose (treat) diabetes type 2. The observed associations between faith-based ownership and competence are qualitatively similar across specifications controlling for various sets of staff-, facility-, and geography-related characteristics.

A potential explanation of the observed competence differences by ownership and medical case may be systematic differences in actual patient case-mix. Government facilities may have a higher proportion of patients who are children displaying symptoms of diarrhoea and severe dehydration, whereas CHAM facilities may have relatively more adult diabetes patients. Otherwise, we do not find any differences in healthcare worker competence for the remaining six cases.

The lack of compelling evidence of an overall difference in clinical competence between providers managed by faith-based-, compared to government organisations, is in line with previous results for Tanzania.¹⁶ Our investigation further reveals heterogeneity by ownership across medical conditions. The findings in our study do not provide an explanation of the previously reported higher comparative patient satisfaction at FBPs relative to public providers.^{5, 12, 13} Keeping in mind the caveats concerning the use of patient satisfaction as an indicator for quality of care, the discrepancy in clinical competence and perceived quality of care by patients may be explained by the “know-do gap”, i.e., the difference between what a clinician knows and what they do in practice.¹⁰

Greater competence measured by vignettes has been shown to be associated with higher effort but should, nonetheless, be interpreted as an upper bound of actual provider behaviour as knowledge of medical protocols does not necessarily mean that the clinician will adhere to the stated protocol during interactions with their usual patients.^{8, 9, 17} The literature has often found that the know-do gap is smaller among non-public and decentralised providers compared to public providers.¹⁰ FBPs in Tanzania exert significantly higher effort during client consultation observations compared to public providers despite no differences in competence.¹⁶

The variation in the know-do gap has been explained by factors related to management, which impact healthcare workers’ intrinsic and extrinsic motivation to exert more effort.¹⁰ In addition, other domains related to structural-and process quality will impact the overall quality of healthcare provision. A study analysing data for multiple African countries report that FBPs fare better on process indicators of quality measured by drug prescription practices and adherence to standard protocols for treatment compared to public facilities.¹² FBPs in Kenya scored the highest on a service readiness index representing the provision of maternal and child health services and good management practices among all managing healthcare organisations, although just marginally higher than government facilities.²⁰ Moreover, compared to other types of healthcare providers, faith-based and publicly managed facilities in Malawi have a lower presence of substandard and falsified drugs.²¹

On the other hand, FBPs in Malawi, Kenya, and Haiti were found to be less likely to offer family planning services compared to differently owned providers,²² and recent evidence shows that Malawian FBPs are less likely to adhere to national sexual and reproductive guidelines concerned with STI prevention during family planning visits.²³ Thus, differences in motivation by organisation and area of health may also impact service provision and performance. However, the majority of research on comparative behavior of faith-based and public provision with respect to process quality rely on direct clinician observations which may be subject to bias stemming from the Hawthorne effect.²⁵ Similarly, a limitation with our study is that we only observe clinical competence in a hypothetical setting and not actual provider behaviour. Moreover, we are unable to estimate the causal impact of faith-based ownership on clinical competence due to the reliance on cross-sectional data. Therefore, we are not able to distinguish whether our findings stem from organisational differences, or the selection of staff as has been reported to be the case in Rwanda and Ethiopia.²⁴

Nonetheless, we show that the observed differences in clinical competence by ownership concerning childhood diarrhoea and diabetes type 2 in adults, as well as the lack of a difference across other common illnesses, are qualitatively similar when adding various controls for provider performance. These findings contribute to the limited knowledge regarding differences in service provision of differently managed healthcare organisations in LMICs. In particular, we add to the scarce literature concerning quality differences between faith-based and public healthcare providers. Rigorous evidence in this area is of great importance as FBPs play an important role in healthcare delivery in SSA, particularly in fragile states, and heterogeneity in service quality may lead to unequal access to quality healthcare and health outcomes. Moreover, governments in SSA often perceive faith-based providers as substitutes to government facilities where public investment in healthcare is lacking, which warrants a thorough investigation in variation in performance.

Competing Interests

None declared.