Introduction
This project was motivated by the experience of two doctors who restarted the then closed-down Makunda Christian Leprosy and General Hospital1 in 1992 and transformed the hospital into a model of high quality, accessible healthcare in a low-resource setting.2 The hospital is located in a remote rural part of Assam, India. Over the next 30 years, the hospital grew into a large, 200-bed institution with poor-centric strategies that brought high capacity utilization to its services. In 2019-20 (the last fiscal year unaffected by COVID-19), the hospital saw 126,382 outpatients, admitted 15,297 inpatients, performed 8,710 surgeries, and conducted 6,750 deliveries. In addition, the Makunda Christian Higher Secondary School, started in 2004, taught 1,300 students from local communities and upgraded its School of Nursing to a College of Nursing. A branch hospital was also started in the neighboring state of Tripura (the first Christian mission hospital in that state).
The early years at Makunda were difficult.1 There were no patients, no external financial support, local opposition, staff rebellion, and the need to provide high quality services safely despite numerous constraints. Since electrical generators had been sold by previous staff to pay salaries, the hospital staff lived without electricity in their homes for the first 14 years, performed surgeries like cholecystectomy without electrocautery, and gave anesthesia with an Epstein-Mackintosh-Oxford apparatus as oxygen and electricity were difficult to procure. Further, due to multiple perceived strains on personal and professional life, it was difficult to recruit and retain professional staff, resulting in heavy workloads on the remaining staff. Although the hospital is doing well 30 years later, the high levels of stress the staff endured might have been avoided and greater impact achieved for patients earlier if additional financial support and manpower were available during the early years.
Hospitals serving the neediest populations across the world face similar scenarios to Makunda in its early years.3 When these hospitals are in the early stages of development and at their most vulnerable, they are invisible to donors and committed staff and volunteers. Donor and human resource directing agencies send support to institutions that have already established impact many years after their establishment. Faced with financial and workload stress, founding staff at hospitals earlier in their impact journey may decide to abandon the work, although hospitals in these locations of great need may have the highest potential for transformational impact.
The impact of healthcare institutions can be incremental or transformational. Healthcare impact is incremental when the hospital’s establishment improves safety, quality, costs (including indirect costs such as travel), and convenience to people who are already accessing healthcare services. Healthcare impact is transformational when the services are designed to enable people who have never accessed modern healthcare services to do so. Further, in many countries in the developing world, healthcare access and outcomes vary tremendously between and within their states. Hospitals that are located in the neediest areas and adopt best practices that enable them to run most effectively and efficiently, as well as improve the personal and professional lives of staff, have the highest level of potential for transformational impact. While impact may take many years to realize, assessing the potential for impact in the early years when these institutions are at their most vulnerable stages of growth may provide access to resources to accelerate their growth. Relatively small investments of funds, manpower, and infrastructure in hospitals with high potential for transformational impact should result in disproportionately large improvements in healthcare indices.
The research objective of this study was to develop a system to evaluate the potential for transformational impact of hospitals that may otherwise remain invisible. The Transformational Impact Rating System (TIRS) rates hospitals based on location and performance factors. The resulting scores can assist donors and potential volunteers and staff in finding hospitals that fit their search criteria and have the highest potential for transformational impact. The increased visibility provided by TIRS could result in increased investment in these hospitals and, therefore, improvements in healthcare indices in the surrounding communities.
Methods
A team of students from the Wharton School of Business conducted the research and produced an initial report on the rating system upon which this article is based.4 Research to inform the rating system was conducted using three primary methods: 1) comprehensive review of existing charity rating systems, 2) in-depth interviews, and 3) surveys. The rating system was developed and implemented as a website for future testing.
1. Review of Existing Systems
A comprehensive review of existing charity rating systems was conducted to identify the strengths and limitations of currently available rating systems. First, an internet search was conducted to identify reputable charity rating organizations.5-15 Methods adopted by each organization were evaluated, including the charities it chooses to rate, the factors it considers in determining a rating, the level of transparency it provides into the rating, and its reliance on information from other charity rating systems. Table 1 summarizes the methods of existing charity rating systems.
Table 1
Methods of Existing Charity Rating Systems
| Organization | Rates All Charities | Measures Impact Based on ≥10 Metrics | Provides Transparency to Donors and Volunteers | Evaluates Charities Independently from Other Charity Rating Systems |
|---|---|---|---|---|
| Charity Navigator | ✗ | ✓ | ✓ | ✗ |
| CharityWatch | ✗ | ✓ | ✓ | ✓ |
| ImpactMatters | ✗ | ✗ | ✓ | ✓ |
| GuideStar | ✓ | ✗ | ✓ | ✓ |
| GuideWell | ✗ | ✗ | ✓ | ✓ |
| GlobalGiving | ✓ | ✓ | ✗ | ✓ |
| Classy | ✓ | ✓ | ✗ | ✓ |
| Universal Giving | ✗ | ✓ | ✓ | ✓ |
| RateYourNGO | ✓ | ✓ | ✗ | ✓ |
| The Life You Can Save | ✗ | ✓ | ✓ | ✗ |
| Bright Funds | ✗ | ✓ | ✓ | ✗ |
2. In-depth Interviews
A total of 12 in-depth interviews were conducted, including 7 representatives of donation management and financial certification organizations and 5 representatives of volunteer and staff sourcing organizations (see Supplemental Table S-1 for details). At the beginning of each interview, interviewees were informed of the purpose of the research and that their responses would be aggregated to inform the project. Semi-structured interviews focused on specific factors that donors and volunteers might consider when choosing to contribute or not contribute to a charitable organization and how they might utilize a rating system (see Supplemental Table S-2 for the full list of interview questions).
3. Surveys
A survey was conducted to evaluate which factors were most important to volunteers in deciding where to volunteer. Survey recipients were identified through interviews with organizations that coordinate volunteers and prior volunteers at Makunda Hospital. The survey asked respondents to rate 28 factors identified through the research as relevant to volunteers on a scale from 1 (not important at all) to 10 (extremely important). In addition, the survey asked respondents to provide any other factors they considered important. A total of 32 responses were received (Supplemental Table S-3 summarizes the results).
4. Rating System
Based on steps 1 through 3 above, an impact rating system was developed to score the transformational impact potential of charitable organizations, specifically focusing on hospitals in low-resource settings. More than 150 variables were identified that could be used to measure impact potential; of these variables, 93 independent variables were selected that repeatedly appeared in the research, were previously validated, were important to donors’ and volunteers’ decision-making based on interviews and surveys, and could be objectively measured. Each variable was given a weight from 1 to 10, with regional and local factors, factors that appeared in the research more frequently, factors that were previously validated, and factors rated as more important in surveys receiving a higher weight. Levels from 1 (low impact potential) to 5 (high impact potential) were defined to measure each variable based on published benchmarks and research2,5-35 as well as the interview findings.(1)
Variables were categorized into one of five categories that emerged: regional impact potential, financial health, operational performance, external relations, or volunteer experience. Variables within the three largest categories (financial health, operational performance, and volunteer experience) were further categorized into subcategories. Using the predefined levels and weights, a hospital receives an integer score from 1-5 for each variable, and weighted averages produce subcategory, category, and overall scores. The final variables included in the rating system with their definitions, categories, and weights are shown in Supplemental Table S-4, and variable levels are shown in Supplemental Table S-5.
5. Website Implementation
The rating system was implemented via a website ( www.transformationalimpact.org) to enable future testing and evolution by simplifying data collection. In the future, the website could also be used to improve the ability of donors and volunteers to find hospitals that have obtained an impact score. To prioritize access and speed to implementation, a mobile-friendly website that could be accessed via any device was chosen over a mobile app, which would need to be designed for multiple operating systems.
First, data collection templates were prepared to define specific data elements that would be collected from the hospital, donors, volunteers, staff, patients, the local community, and publicly available sources. For the first iteration, the hospital data collection template was implemented on the website by allowing a user to sign in via Google account, add their hospital to the system, and answer questions for each variable by selecting one of the predefined levels for each variable. A sample of the questions is shown in Figure 1. All questions are required. (The user may select “No Data”; however, selecting this option results in a score of 0 for that variable to disincentivize hospitals from reporting “No Data” in areas where they may be underperforming.) The website then calculates the hospital’s score as described in Results, below. Users may check their overall rating before submitting their data and allowing their score to be publicly displayed. Information that can be populated from public sources is pulled in via web scraping.
Figure 1
Sample Hospital Data Collection Questions
From the home page of the website, visitors can see hospitals that have been rated, their overall scores, and their locations on a map. By selecting a hospital, visitors are directed to the hospital’s page which includes a description of the hospital, causes it supports, its overall score, and scores for each category and subcategory, as shown in Figure 2. Future iterations of the website could allow donors and volunteers to see the variables that are of interest to them, provide feedback on hospitals to which they have previously donated or at which they have volunteered, and see comments from other users.
Figure 2
Sample Category Scores
6. Proof-of-Concept Testing
As a proof of concept, the rating system was initially tested using hypothetical data for two hospital archetypes (one that might be considered high impact and one that might be considered low impact) as well as with actual Makunda Hospital data. Data for these hospitals were input into the data collection templates and the resulting scores were examined to determine whether the scores accurately reflected the hospital’s expected transformational impact potential. Levels were adjusted for some variables and some variables were removed based on the results of these tests. The final results of the tests were an overall score of 4.1 for the archetype of a hospital with relatively high transformational impact potential and room for improvement; an overall score of 2.6 for the archetype of a hospital with relatively low transformational impact potential but high resources and average performance; and an overall score of 4.6 for Makunda Hospital, which has been studied for its high transformational impact.2
Results
1. Data Collection
To obtain a TIRS impact score, data for each variable must first be collected for the hospital. The hospital provides information for 69 variables that evaluate the hospital’s regional impact potential, financial health, operational performance, external relations, and volunteer experience. Although collecting data on so many variables could be burdensome for hospitals, many variables require qualitative assessment rather than quantitative calculation. In addition, all data is collected by selecting one of the five levels for each variable, such that understanding the range the data falls within is sufficient. Both of these functional choices reduce the burden of data collection for hospitals.
In addition to collecting data from the hospital, data is collected from surveys of donors, volunteers, staff, patients, and the local community. Volunteers are asked to rate two variables related to the hospital’s commitment to its mission and the credibility of its leadership, six variables related to the logistics of the volunteer opportunity, and thirteen variables related to their satisfaction with their experience. Staff are asked to rate five variables related to the hospital’s commitment to its mission, level of community engagement, culture of continuous improvement, the credibility of its leadership, and the staff’s level of satisfaction. Staff are also asked to score three variables related to volunteer opportunities. Donors, patients, and the local community are asked to rate only three variables each, focusing on their satisfaction and the hospital’s performance. In each survey, respondents may provide additional comments that can be shared with those viewing the hospital’s score.
Finally, data that is publicly available from the internet is collected to rate some variables. For the first iteration of the rating system, the only variable that relies solely on data from the internet is whether the organization has previously been rated by another charity rating system. While country-level data on maternal mortality rate, infant mortality rate, population density, and crime rates are provided to the hospital for reference, hospitals select the levels for those local, non-modifiable factors due to lack of more granular publicly available data within a country.
2. Calculating the Score
Once data has been collected for each variable, the hospital receives an overall score from 1 to 5, where 5 represents a hospital with the highest possible impact potential. This score is simply a weighted average of each variable’s rating (based on the levels and weights discussed in the Methods and detailed in Supplemental Tables S-4 and S-5). Variables for which no data is available receive a score of zero (rather than being excluded from the score) to encourage future collection of these data.
In addition to the overall score, hospitals receive a score in each of five categories: regional impact potential, financial health, operational performance, external relations, and volunteer experience. Category scores allow hospitals to understand on a broad level in which areas they are performing well and in which areas they could improve. These category scores are simple weighted averages of the ratings of the variables that fall within the category. In other words, specific categories are not given additional weight; the importance of the category was already considered when weighting individual variables.
Regional impact potential represents the hospital’s ability to make an impact on the community based on local needs. This category largely consists of factors over which the hospital has no control. Financial health represents the extent to which the hospital uses its funds efficiently, manages its funds properly, and is sustainable. Operational performance is the largest category, including factors that measure how efficiently the hospital executes its mission, its level of governance, its transparency, the number of patients the hospital serves, and how well it serves its patients. External relations measures how well the hospital collaborates with other parties, such as donors, the community, and other programs. Finally, volunteer experience measures the quality of the experience, logistics, and satisfaction of prior volunteers.
Because financial health, operational performance, and volunteer experience encompass the most factors, each category is broken down further into subcategories. Financial health includes funding sources, financial efficiency, financial capacity, and accountability. Operational performance includes operational efficiency, governance, transparency, organization quality, volume of services, and care quality. Volunteer experience includes volunteer opportunity and volunteer satisfaction. Hospitals receive a score in each subcategory (calculated as a weighted average of the variables included in the subcategory) so that they can identify more granular areas in which they can either continue to perform well or improve.
Figure 3 shows a diagram of the scoring hierarchy.
3. Using the Score
Once data has been collected and TIRS scores have been calculated, hospitals can view their overall score, category scores, and subcategory scores, as well as ratings for each individual variable. Hospitals can view which factors are modifiable (i.e., can be changed based on the hospital’s performance) and which are non-modifiable (i.e., are based on local conditions). By examining the modifiable factors for which they scored the lowest, hospitals can review the criteria for obtaining a higher score and take actions to improve their rating (starting with the modifiable variables which are weighted most highly and / or require little effort to improve) with the goal of increasing their attractiveness to donors and volunteers. Hospitals may choose to update their data at least annually, or sooner if they make significant changes, to ensure donors and volunteers can view the most up-to-date score.
In the future, donors and volunteers could choose to contribute to hospitals with the highest TIRS scores that match their interests. Users could search the TIRS website based on TIRS scores and additional factors (noted at the end of Supplemental Table S-4), such as the geography in which the hospital is located, any specific causes the hospital supports (e.g., maternal health, leprosy), whether the hospital has a Christian mission, whether the hospital can accept foreign donations, and the ease with which volunteers can obtain visas. Hospitals that meet the user’s search criteria would be displayed, and hospitals that have the highest overall impact score would be shown at the top. Users could select each hospital to view more information, such as category and subcategory scores, as well as any descriptive information the hospital has provided.
4. Score Differentiation
Many charity rating systems exist today, as summarized in Table 1. However, many organizations choose which charities to rate and do not accept requests from charities that ask to be rated. Even organizations that accept charities’ requests to be evaluated may limit their evaluation to charities in America that have IRS filings, or they may have strict criteria that the charity must meet in order to be rated. For example, Charity Navigator only rates US-based 501(c)(3) tax-exempt organizations.5 As a result, small but high impact hospitals in needy parts of the world have no presence on these rating websites. In contrast, TIRS is available to any hospital that wishes to be rated, regardless of size, operating metrics, or location.
In terms of the target audience of these rating systems, nearly all (with the exception of Universal Giving) focus on donors and do not address specific factors of interest to volunteers. However, many high impact hospitals in low resource settings could benefit nearly or equally as much from volunteer doctors who can help them serve their communities as from monetary donations. To fill this gap, TIRS includes factors that are important to both donors and volunteers, so that volunteers can find hospitals where volunteering can have the greatest impact.
Additionally, existing rating systems ignore regional impact factors. They evaluate charities based on how efficiently they use their funding, how well they have sustained their programs and services over time, and their level of commitment to good governance, best practices, and openness with information. For instance, CharityWatch considers a charity that spends at least 75% of their funds on program expenses and that spends no more than $25 to raise each $100 to be highly efficient.6 Consequently, the ratings represent how well charities currently perform in terms of financial health, operational performance, transparency, and governance. However, current systems fail to consider the regional impact potential of organizations located in areas where the populations served have particularly great needs. Because regional impact factors measure the need of the community and directly measure an organization’s potential for impact, TIRS includes these factors in addition to validated factors included in existing rating systems.
Discussion
Considering the strengths and weaknesses of existing rating systems, TIRS was developed to provide visibility to hospitals with high potential for transformational impact. To address the issue that small organizations and organizations outside of the U.S. are usually excluded from existing rating systems, TIRS will be open to all legally registered hospitals. Hospitals are not required to meet any criteria on revenue, length of operation, or otherwise. To fill in the gap of impact factors relevant to volunteers, TIRS captures information related to volunteer satisfaction and opportunities. Additionally, TIRS rates regional impact factors so that hospitals’ potential for community impact can be measured along with their existing performance. TIRS is designed to enable donors and volunteers to find opportunities where their contributions would be most impactful.
TIRS in its current state is a proof of concept that demonstrates data-driven decision-making for funding and staffing hospitals with high potential for transformational impact. The system should continue to be improved and can be customized to meet specific needs. Potential applications and modifications include:
Refine and simplify TIRS. TIRS currently includes 93 variables, making data collection burdensome. Opportunity exists to test TIRS with the actual hospital data, simplify data collection by combining or reducing variables, and automate collection of publicly available data to improve the usability and accessibility of TIRS.
Create customized versions of TIRS for specific geographies. Because the current scope of TIRS is global, some variables have very wide data ranges. For example, a maternal mortality rate of 600 currently receives a score of 5, indicating great need; however, this rate would be low for South Sudan and high for India. Versions customized for specific countries could use data ranges that are more relevant for a given geographical location.
Implement third-party verification. The present version of TIRS is based largely on hospitals’ self-assessment, which introduces bias. Administrators can employ third-party evaluators to visit and periodically assess hospitals to verify the data provided and improve credibility. The scores of hospitals that have been evaluated by third parties can be indicated as “verified.”
Provide hospitals with individual impact reports. Individual detailed reports, including which factors are modifiable and actionable recommendations, can help hospitals improve their score. Adoption of TIRS by major donors and human resource directing agencies coupled with individualized reports could be a powerful tool for institutional behavior change.
Implement additional data collection on the website. While TIRS collects data from surveys of donors, volunteers, staff, patients, and the local community, the website implementation of TIRS currently only collects hospital-reported data. Future versions of the website could implement these additional surveys. Furthermore, when donors and volunteers engage with a hospital, they could be invited to provide their assessment at the end of their engagement.
Improve website searchability. Tags could be added to hospital profiles so that hospitals can be searched based on location (e.g., Western Africa), service profile (e.g., maternal health), denominational background (e.g., Christian), or other factors that may be of interest to donors, volunteers, and staff.
Customize TIRS over time. The factors and weights for each factor can be customized as the internal and external environment in a particular situation or application evolve over time. Site administrators can be provided access to granular information from hospitals that allow fine-tuning of factors to suit emerging needs.
Create customized versions of TIRS for specific uses. TIRS can be used for other impact areas of interest, such as agricultural or educational institutions, by substituting healthcare-related factors for appropriate factors in these areas.
Conclusion
In the neediest parts of the world, healthcare investment by government entities is inadequate and interest from private donors and volunteers is insufficient. A primary challenge that hospitals in these areas face is limited access to financial and human capital. Many charity rating systems only rate a limited number of organizations or do not provide sufficient transparency into the ratings they provide. The objective of this project is to develop a rating system through which all hospitals in low-resource settings can make their impact potential visible to donors and volunteers looking to make the greatest difference with their money and time.
The TIRS rating system combines information on 93 variables across five categories to provide hospitals with an overall impact score (as well as scores by category and subcategory). To obtain a score, information is collected from a variety of sources, including the hospital, publicly available information, and surveys of donors, volunteers, staff, patients, and the community. Donors and volunteers can use these ratings to identify hospitals with high impact potential where their donations and time will make the greatest difference. In addition, hospitals can use detailed information from their scores to continue performing well and to improve in areas that could allow them to access additional resources.
TIRS demonstrates data-driven identification of hospitals with the greatest potential for transformational impact, especially for donors and volunteers. If TIRS (with suggested modifications, such as simplification, customization, and verification) is used by those interested in helping mission hospitals, these hospitals will be more likely to adopt best practices, and invisible hospitals struggling in remote areas will be more likely to receive much-needed resources. Ultimately, the goal of TIRS is to connect small, non-profit hospitals with appropriate resources to improve the lives of those who need their services most. TIRS and derived tools can be powerful instruments in the hands of entities and individuals that seek to revive sick mission hospitals across the globe and establish new ones by using data to direct resources to hospitals that have the highest potential for transformational impact.
Competing Interests
None declared
Acknowledgements
We thank the staff of many institutions (anonymously listed in Supplemental Table S-1) who provided time for interviews. We acknowledge the assistance provided to this project by Cindy Li, Chip Stine, and Lydia Lu, all MBA students at the Wharton School of Business who were part of this study and based in the USA. We also acknowledge the website development support provided by Angela Lin, Brendan McCaffrey, and Kat Wang, all students at the School of Engineering and Applied Science at the University of Pennsylvania. We thank Dr. Nalli Chandan, CEO of Makunda Christian Hospital, for filling in the test questionnaire for that hospital. We also thank Dr. Stephen Sammut, faculty member at the Wharton School of Business, for his review of the final report.
Technical Implementation
The website was built primarily using serverless technology to minimize maintenance costs and dynamically scale resources to match user traffic. Multiple Firebase (Google Cloud) products were used to support the platform. All the data was stored in Firestore, a NoSQL database. Firebase Authentication was used to securely add hospital data. Passwords were encrypted using Google Auth for security. Firebase's Security Rules were used to maintain a proper access control list, maximizing user privacy. Finally, HTML, CSS, and client-side JavaScript were used to develop the front end of the website. All the code is rendered client-side with documentation for transparency and ease of access.
Supplemental Table 1
Table S-1
Interview List
| Serial # | Title | Organization Type |
|---|---|---|
| 1 | Executive Vice President | Global Donor-Advised Fund |
| 2 | Chief Operating Officer | US-based Donor Collaborative |
| 3 | Chairman | Charity Financial Certification Body |
| 4 | Executive Director | Organization Development Institute |
| 5 | Partner | Non-profit Donor Organization |
| 6 | CEO/President | Impact Investment Organization |
| 7 | Secretary/Board Member | Charitable Investment Foundation |
| 8 | Founder | Volunteer Coordination Organization |
| 9 | President | Healthcare Worker Training Provider |
| 10 | Development Coordinator | Healthcare Worker Training Provider |
| 11 | General Secretary | Medical Fellowship |
| 12 | Prior CEO | International Medical Association |
Supplemental Table 2
Table S-2
Sample Interview Questions
| Question # | Question |
|---|---|
| 1 | Please describe the organization you represent and its role in making donations / coordinating volunteers. |
| 2 | What are the primary factors you or those you work with consider when choosing where to donate / volunteer? These could be particular causes, motivations, or aspects of the charitable organization. |
| 3 | What is the process you use to find an organization to donate to / volunteer at (e.g., word of mouth, research by topic, recommendation websites, etc.)? |
| 4 | What types of communications are most effective in informing your decision to donate / volunteer (e.g., online sources, mail, in-person events)? |
| 5 | What are some factors that might prevent you or those you work with from donating to / volunteering at an organization, or lead you to choose one organization over another? |
| 6 | What factors determine whether you or those you work with donate to / volunteer at an organization repeatedly versus one time? |
| 7 | If there were a website that allowed you to search for high impact hospitals in locations where contributions would make the greatest difference, what features would you like to see? What aspects would you like to search for hospitals based upon (e.g., location, cause, safety, etc.)? |
| 8 | How would an organization’s impact rating (e.g., 1 star to 5 stars) influence your decision or that of those you work with to donate to / volunteer at that organization? |
| 9 | Would you or those you work with be interested in completing a survey about your donation / volunteer experience on the website that would factor into the hospital’s rating? |
Supplemental Table 3
Table S-3
Volunteer Survey Questions and Responses
| Question | Mean Score | Standard Deviation |
|---|---|---|
| #1: Values / Philanthropy (volunteers' satisfaction with opportunities to express their values of related to altruistic and humanitarian concerns for others) | 8.78 | 1.92 |
| #2: Understanding / Learning (volunteers' satisfaction with opportunities to learn new things and exercise their knowledge and skills) | 8.35 | 1.38 |
| #3: Career Development (volunteers' satisfaction with career development opportunities provided by the program, such as level of training received, preparation for future employment, development of network or leadership skills) | 7.06 | 2.02 |
| #4: Social / Affiliation (volunteers' satisfaction with the level of personal interaction, ease of making friends, ability to work in a group, and ability to develop trust) | 7.61 | 1.93 |
| #5: Achievement (volunteers' satisfaction with having specific goals to work toward, level of responsibility, and challenges to solve) | 7.65 | 1.60 |
| #6: Influence (volunteers' satisfaction with their ability to impact and influence decisions) | 7.43 | 2.04 |
| #7: Job Training Appropriateness (volunteers' satisfaction with how appropriate training was for the tasks they were asked to complete) | 7.37 | 1.70 |
| #8: Recognition (volunteers' satisfaction with friends' or colleagues' recognition of their work) | 6.20 | 2.12 |
| #9: Supervision (volunteers' satisfaction with the level and quality of supervision they received) | 7.13 | 2.00 |
| #10: Volunteer Recommendation (whether previous volunteers would recommend volunteering at this hospital) | 7.67 | 2.21 |
| #11: Expected Volunteer Experience (number of years and type of experience volunteers need to be successful at the hospital) | 6.30 | 1.70 |
| #12: Expected Volunteer Responsibility (volunteers’ responsibility in terms of hours and difficulty) | 6.80 | 1.80 |
| #13: Volunteer Salary (financial compensation) | 5.03 | 2.47 |
| #14: Spousal Opportunities (availability of opportunities at or near the hospital for volunteers’ spouses) | 6.67 | 2.39 |
| #15: Number of volunteers that previously served the hospital | 5.17 | 2.00 |
| #16: Percentage of volunteers that returned to volunteer at the same hospital | 5.77 | 2.17 |
| #17: Environment / Wildlife (how appealing the natural environment is to potential volunteers) | 5.97 | 2.27 |
| #18: Local Safety | 7.35 | 1.99 |
| #19: Visa Requirement (how difficult it is for volunteers to obtain a visa) | 6.65 | 1.71 |
| #20: Specific volunteer needs the hospital has | 7.45 | 1.43 |
| #21: Specific cause the hospital supports | 7.97 | 2.06 |
| #22: Regional Impact Potential (e.g., patient population and square mileage covered by the hospital, health status and socioeconomic status of patient population) | 8.00 | 1.82 |
| #23: Transparency of the Hospital (e.g., does the organization have a website with information pertinent to donors and volunteers?) | 7.23 | 2.34 |
| #24: Volume of Healthcare Services (e.g., number of outpatient visits, inpatient admissions, major surgeries, child deliveries) | 7.42 | 1.93 |
| #25: Efficiency of Operational Performance (e.g., cost per life saved, average cost per patient) | 7.10 | 2.28 |
| #26: Quality of Operational Performance (e.g., patient mortality rate, patient satisfaction, physician performance, staff quality, quality certifications) | 7.07 | 1.88 |
| #27: Hospital’s relations with the local community, staff, and competitors | 7.31 | 2.10 |
| #28: Number of hospital page views on the ratings website (or other interactions such as comments/reviews/recommendations) | 4.21 | 2.04 |
Supplemental Table 4
Table S-4.a
Variable Descriptions and Weights
| Category | Subcategory | Variable | Description | Weight |
|---|---|---|---|---|
| Regional Impact Potential | Regional Impact Potential | Socioeconomic Status of Patient Population | Into which percentile of the country's income per capita does the average patient population that the hospital serves fall? The average patient income can be estimated based on all sources of income (including wages, proceeds from sales, etc.). This tool can be used for reference: https://wid.world/simulator. | 10 |
| Regional Impact Potential | Regional Impact Potential | Local Population Income Mix | Does a mix of incomes exist in the region such that wealthier patients can be charged more to offset charity care for lower income patients? In other words, what is the standard deviation of the local population income? | 6 |
| Regional Impact Potential | Regional Impact Potential | Local Patient Population Density | What is the local population density within serving distance of the hospital (people per square kilometer)? | 8 |
| Regional Impact Potential | Regional Impact Potential | Alternative Care Options | How easy is it for the patient population to receive care at another hospital? | 9 |
| Regional Impact Potential | Regional Impact Potential | Local Maternal Mortality Rate (MMR) | What is the maternal mortality rate (number of deaths of maternity patients per 100,000 live births annually) in the region where the hospital is located? | 10 |
| Regional Impact Potential | Regional Impact Potential | Local Infant Mortality Rate (IMR) | What is the infant mortality rate (number of deaths of infants per 1,000 live births annually) in the region where the hospital is located? | 10 |
| Regional Impact Potential | Regional Impact Potential | Additional Services | What services does the organization provide (e.g., hospital, education, agriculture)? Please list each as a way to allow donors and volunteers to search for the organization. | 4 |
| Financial Health | Funding Sources | Diversity of Revenue Sources | How many sources of revenue does the hospital have (as a measure of sustainability)? | 7 |
| Financial Health | Funding Sources | Donations Percent of Revenue | What percentage of the hospital's annual revenue is provided by donations (as a measure of sustainability)? (To calculate, add the value of all donation sources and divide by the annual revenue.) | 7 |
| Financial Health | Financial Efficiency | Operating Margin | What percent of annual revenue is the hospital able to retain after costs are paid (as a measure of sustainability)? | 6 |
| Financial Health | Financial Efficiency | Program Expense Percentage | What percent of expenses consist of program expenses (e.g., non-administrative and non-capital expenditures)? (To calculate, divide the average program expense by the average total expense, over three most recent fiscal years.) | 7 |
| Financial Health | Financial Efficiency | Administrative Expense Percentage | What percent of expenses consist of administrative expenses (e.g., human resources, accounting, overhead)? (To calculate, divide the average administrative expense by the average total expense, over three most recent fiscal years.) | 5 |
| Financial Health | Financial Efficiency | Capital Expenditure Percentage | What percent of expenses consist of reinvesting in capital expenditures (e.g., new buildings / equipment)? (To calculate, divide the average capital expenditure by the average total expense, over three most recent fiscal years.) | 5 |
| Financial Health | Financial Efficiency | Fundraising Expense Percentage | What percent of expenses consist of fundraising expenses? (To calculate, divide the average fundraising expense by the average total expense, over three most recent fiscal years.) | 5 |
| Financial Health | Financial Efficiency | Fundraising Efficiency | How much does it cost the hospital to bring in each $1 of donations? (To calculate, divide the average fundraising expense by the average total donations, over three most recent fiscal years.) | 5 |
| Financial Health | Financial Capacity | Program Expense Growth | By how much have program expenses grown annually over the three most recent fiscal years? (To calculate: [(Expense in year 3/Expense in year 1)^(1/3) - 1].) | 4 |
| Financial Health | Financial Capacity | Liabilities to Assets Ratio | What is the hospital's liabilities to assets ratio (as a measure of sustainability)? (To calculate, divide total liabilities by total assets in the most recent fiscal year.) | 1 |
| Financial Health | Financial Capacity | Charity Care Percentage | What percentage of annual revenue is given to patients as charity (free) care? | 6 |
| Financial Health | Accountability | Budget | Does the hospital prepare a budget and have adequate processes of budgetary controls? Does the hospital prepare a budget variance analysis? | 2 |
| Financial Health | Accountability | Auditing | Does the hospital have audited financial statements prepared by an independent accountant with an audit oversight committee? | 1 |
| Financial Health | Accountability | Accounting Books | Does the hospital maintain proper books of accounts? | 1 |
| Financial Health | Accountability | Bank Accounts | Are the hospital's bank accounts in the name of the hospital, not an individual? | 1 |
| Financial Health | Accountability | Safety of Funds | Is there any evidence of lost or misused donation dollars, fines, fraud, or the like? | 5 |
| Operational Performance | Operational Efficiency | Cost Per Life Saved | How much does the hospital spend per lives saved annually? (To calculate, divide total program and administrative expenses by the number of lives saved.) | 7 |
| Operational Performance | Operational Efficiency | Charity Care Efficiency | How efficiently does the hospital allocate charity care dollars (as a measure of accessibility)? For example, does the hospital use an ability-to-pay pricing approach, write off unexpected complications, and/or assess charity care needs using behavioral methods such as the “shared meals” and “vital assets” tests. | 6 |
| Operational Performance | Operational Efficiency | Average Inpatient Cost | What is the average price charged to inpatients (the cost of the stay less any charity care), as a percent above the national average? Average costs can be estimated by selecting your country in the WHO spreadsheet at the following site: https://www.who.int/choice/country/country_specific/en/ | 5 |
| Operational Performance | Operational Efficiency | Average Outpatient Cost | What is the average price charged to outpatients (the cost of the visit less any charity care), as a percent above the national average? Average costs can be estimated by selecting your country in the WHO spreadsheet at the following site: https://www.who.int/choice/country/country_specific/en/ | 5 |
| Operational Performance | Governance | Formal Governance Structure | Does the hospital have a formal governance structure established? | 6 |
| Operational Performance | Governance | Commitment to Mission | How committed and dedicated to the mission of the hospital are staff and volunteers? | 3 |
| Operational Performance | Governance | Strategic Plan and Implementation | Does the hospital have a strategic plan to improve and expand services? Are planned strategies and interventions implemented properly? | 6 |
| Operational Performance | Governance | Key Performance Indicators | Are key performance indicators defined and in place to monitor progress with respect to inputs, activities, outcomes and impacts? | 3 |
| Operational Performance | Governance | Alternate Implementation Plans | Are the feasibility and cost-effectiveness of alternate implementation plans discussed? | 3 |
| Operational Performance | Governance | Systems Capacity | Do the hospital's systems have sufficient capacity, and are they able to scale? | 6 |
| Operational Performance | Governance | Board Independence | Is the hospital's board independent (e.g., board is not comprised of too many staff members)? | 2 |
| Operational Performance | Governance | Permanent Board Members | Does the hospital have board members/trustees that are permanent in nature (with no fixed term) or who have served for 10 years or more? | 1 |
| Operational Performance | Governance | Board Meetings | Are board meetings held regularly? | 1 |
| Operational Performance | Governance | Annual General Meetings | Are annual general meetings held? Are statutory resolutions such as appointment of auditors and election of board members passed during general meetings? | 1 |
| Operational Performance | Governance | Board Review | Does the board review itself, systems, processes, and CEO/staff regularly? | 1 |
| Operational Performance | Governance | Board Donation Oversight | Does the board have oversight over how donated funds are used? | 1 |
| Operational Performance | Governance | Appointed CEO | Does the hospital have an appointed CEO? | 1 |
| Operational Performance | Governance | Legal Support | How strong is the hospital's legal support and competency in analyzing a changing regulatory landscape / suggesting appropriate service providers? | 2 |
| Operational Performance | Governance | Operational Policies | Does the hospital have and follow documented workplace policies (e.g., human resources, travel, finance, conflict of interest, whistleblower, employee use of assets, records retention and destruction)? | 1 |
| Operational Performance | Governance | Culture of Continuous Improvement | Does the hospital have a culture of continuous improvement (e.g., staff have agency, quality control systems are in place)? | 1 |
| Operational Performance | Transparency | Outcomes Reporting | How well and often does the hospital communicate the loss prevented or gains resulting from donations? | 3 |
| Operational Performance | Transparency | Donor Privacy Policy | Does the hospital have a privacy policy that allows donors to request their name and address not be shared with other organizations? | 1 |
| Operational Performance | Transparency | Website | Does the hospital have a website with information pertinent to donors and volunteers? | 1 |
| Operational Performance | Organization Quality | Appropriate Standards of Care | Does the hospital use efficient care standards to ensure services are both affordable and nearly as effective and safe as more expensive alternatives? | 6 |
| Operational Performance | Organization Quality | Equal Services for All Patients | Do all patients receive equal care regardless of their wealth (as a measure of accessibility)? | 7 |
| Operational Performance | Organization Quality | Community Engagement | How well does the hospital work with community experts, partners, staff, and patients to build support, plan for the future, and impact the health of their local communities? | 6 |
| Operational Performance | Organization Quality | Quality Certifications | Has the hospital received safety and quality certifications? | 2 |
| Operational Performance | Organization Quality | Awards Conferred | Has the hospital received awards and recognition from third parties in recognition of the hospital's impact? | 4 |
| Operational Performance | Volume | Major Surgeries | How many major surgeries does the hospital provide (per 1000 population served) annually? | 5 |
| Operational Performance | Volume | Child Deliveries | How many child deliveries does the hospital provide (per 100 women of childbearing age in the population served) annually? | 5 |
| Operational Performance | Volume | Outpatient Visits | How many outpatient visits (per 1000 population served) does the hospital provide annually? | 6 |
| Operational Performance | Volume | Bed Occupancy | What is the hospital's bed occupancy rate? (To calculate, divide total number of bed-days during the year by (number of beds available * 365 days), then multiply by 100.) | 7 |
| Operational Performance | Volume | Weighted Average Patient Volume CAGR | What is the average weighted compound annual growth rate over the past three years of outpatient visits, inpatient admissions, major surgeries, and deliveries? | 4 |
| Operational Performance | Care Quality | Hospital Maternal Mortality Rate (MMR) | How many deaths of maternity patients does the hospital experience (per 100,000 live births annually), as compared to the local maternal mortality rate? | 4 |
| Operational Performance | Care Quality | Hospital Infant Mortality Rate (IMR) | How many deaths of infants does the hospital experience (per 1,000 live births annually), as compared to the local infant mortality rate? | 4 |
| Operational Performance | Care Quality | Patient Mortality Risk | How severe are the conditions of patients seen, as a measure of patient mortality risk? | 2 |
| Operational Performance | Care Quality | Patient Readmission Rate | What percent of patients must return to the hospital and be admitted within 30 days of being discharged? | 1 |
| Operational Performance | Care Quality | Patient Satisfaction - Service | As a patient, how satisfied are you with the hospital's service? | 1 |
| Operational Performance | Care Quality | Patient Satisfaction - Staff | As a patient, how satisfied are you with the hospital's staff? | 1 |
| Operational Performance | Care Quality | Patient Satisfaction - Hospital Cleanliness | As a patient, how satisfied are you with how clean the hospital is? | 1 |
| Operational Performance | Care Quality | Staff Satisfaction | As a staff member, how happy are you with your experience? | 1 |
| Operational Performance | Care Quality | Staff Experience | Does the hospital have staff with work experience of 10 years and above in a particular field? | 1 |
| Operational Performance | Care Quality | Staff Meetings | Does the hospital hold regular staff meetings? | 1 |
| Operational Performance | Care Quality | Staff Turnover | In the past two years, what has been the overall staff turnover rate? | 1 |
| External Relations | External Relations | Leadership Credibility | How credible and accomplished is the hospital's leadership? | 7 |
| External Relations | External Relations | Donor Satisfaction | As a donor, how satisfied are you with your donation to the hospital? | 3 |
| External Relations | External Relations | Charity Rating Organizations | Is the hospital rated by large charity rating organizations? If so, how well is the hospital rated? | 1 |
| External Relations | External Relations | Collaboration | Does the hospital have partners that it is are collaborating with, and does it plan to add partners to its network? | 2 |
| External Relations | External Relations | Exchange Programs | Does the hospital have relationships/exchange programs with other similar organizations or universities? | 1 |
| External Relations | External Relations | Respect of Community | How well does the local community respect the hospital? | 1 |
| Volunteer Experience | Volunteer Opportunity | Volunteer Responsibility | What is the level of responsibility expected of volunteers, in terms of hours and difficulty? | 1 |
| Volunteer Experience | Volunteer Opportunity | Expected Volunteer Experience | How many years and what type of experience do volunteers need to be successful? | 1 |
| Volunteer Experience | Volunteer Opportunity | Spousal Job Opportunities | Are nearby opportunities readily available for volunteers' spouses? | 1 |
| Volunteer Experience | Volunteer Opportunity | Child Education Opportunities | Is quality education available for volunteers' children? | 1 |
| Volunteer Experience | Volunteer Opportunity | Housing Opportunities | How difficult is it for volunteers to find quality housing? | 1 |
| Volunteer Experience | Volunteer Opportunity | Local Safety | How safe is the area in which the hospital is located? | 2 |
| Volunteer Experience | Volunteer Opportunity | Local Safety Perception | How safe is the area in which the hospital is located? | 2 |
| Volunteer Experience | Volunteer Satisfaction | Volunteer Recommendation | As a volunteer, would you recommend volunteering at this hospital? | 3 |
| Volunteer Experience | Volunteer Satisfaction | Overall Volunteer Satisfaction | As a volunteer, how satisfied are you with your experience overall? | 1 |
| Volunteer Experience | Volunteer Satisfaction | Skills Match | As a volunteer, how satisfied are you with how well your skills matched the needs of the hospital and community? | 1 |
| Volunteer Experience | Volunteer Satisfaction | Learning Opportunities | As a volunteer, how satisfied are you with the opportunities for new learning experiences, and opportunities to exercise your knowledge, skills, and abilities? | 3 |
| Volunteer Experience | Volunteer Satisfaction | Career Development | As a volunteer, how satisfied are you with the level of training received, preparation for future employment, the opportunity to develop leadership skills, network development, and references for future employment? | 2 |
| Volunteer Experience | Volunteer Satisfaction | Job Training Appropriateness | As a volunteer, how satisfied are you with how appropriate training was for the tasks you were asked to complete? | 2 |
| Volunteer Experience | Volunteer Satisfaction | Community Affiliation | As a volunteer, how satisfied are you with the level of personal interaction, ease of making friends, ability to work in a group, and ability to develop trust? | 3 |
| Volunteer Experience | Volunteer Satisfaction | Strategy and Influence | As a volunteer, how satisfied are you with your ability to impact and influence decisions, respond to the needs of people or programs, and understand the overall goals of the hospital? | 2 |
| Volunteer Experience | Volunteer Satisfaction | Philanthropic Impact | As a volunteer, how satisfied are you with opportunities to express your values related to altruistic and humanitarian concerns for others and to impact the community? | 3 |
| Volunteer Experience | Volunteer Satisfaction | Flexibility | As a volunteer, how satisfied are you with the flexibility of your work schedule? | 1 |
| Volunteer Experience | Volunteer Satisfaction | Supervision | As a volunteer, how satisfied are you with the level and quality of supervision and feedback you received? | 2 |
| Volunteer Experience | Volunteer Satisfaction | Recognition | As a volunteer, how satisfied are you volunteers with opportunities to be recognized for your contributions by friends and colleagues? | 1 |
| Volunteer Experience | Volunteer Satisfaction | Achievement | As a volunteer, how satisfied are you with having specific goals to work toward, level of responsibility, and challenges to solve? | 3 |
Table S-4.b
Hospital Page Description and Search Criteria
| Category | Variable | Description |
|---|---|---|
| Page Description | Hospital Name | What is the name of your hospital? |
| Page Description | Contact | What is a good e-mail address at which your hospital can be contacted? |
| Page Description | Website URL | If your hospital has a website where donors and volunteers can find additional information, please provide the link to the website. |
| Page Description | Description | Please provide a brief description of your hospital that you would like to be displayed on your hospital’s page. |
| Page Description | Picture | Please provide a picture of your hospital that you would like to be displayed on your hospital’s page. |
| Page Description | Beneficiary Narrative | If you would like, you may provide a story of an individual whose life your hospital has benefited. Personal stories often help donors and volunteers make a personal connection to a charitable organization. |
| Page Description | Bed Capacity | Please provide the number of beds your hospital has for the description on the hospital’s page. |
| Page Description | Annual Patients Served | Please provide the number of patients your hospital serves annually for the description on the hospital’s page. |
| Page Description | Donation Needs | Does the hospital have any specific donor needs? If so, please explain, as a way for potential donors to find the hospital. If available, please provide a plan for future funds, including specific uses (e.g., new program or facility), amounts, priority, and alignment to the hospital’s strategic plan. |
| Page Description | Donation Uses | How has the hospital used donations in the past? |
| Page Description | Volunteer Needs | Does the hospital have any specific volunteer needs? If so, please explain, as a way for potential volunteers to find the hospital. If available, please provide a plan for volunteer work (e.g., specific procedures the volunteer will get to learn or perform, learning objectives). |
| Page Description | Volunteer Needs | Does the hospital have any specific volunteer needs? If so, please explain, as a way for potential volunteers to find the hospital. If available, please provide a plan for volunteer work (e.g., specific procedures the volunteer will get to learn or perform, learning objectives). |
| Page Description | Visa Requirement | How difficult is it for volunteers to obtain a visa to work with the hospital? Please explain. |
| Page Description | Volunteer Salary | Are volunteers able to receive a salary? If so, what is the average salary or a typical range of salaries? |
| Page Description | Attractions | Are there any features of the hospital’s location that you would like to highlight that might attract short-term volunteers, such as music, wildlife photography, nearby tourist attractions, etc.? |
| Page Description | Location Access | Please describe the location in which the hospital is located and how difficult it would be for volunteers to travel there. |
| Search Criteria | Geography | In which country is the hospital located? |
| Search Criteria | Cause | Does the hospital support any specific causes (e.g., leprosy, cancer, maternal health)? Please list each as a way to allow donors and volunteers to search for the hospital. |
| Search Criteria | Christian Mission | Does the hospital have a Christian-based mission? |
| Search Criteria | Ability to Accept Foreign Donations | Is the hospital registered with the appropriate bodies required to receive foreign donations (e.g., FCRA in India)? |
| Search Criteria | Conditional Donations | Can the hospital accommodate donations that come with conditions tied to them? |