Table 1
Authorship of the Sample of Media Assistance Evaluation Reports
| Authored by | Total | Commissioned (/ required) by donor | Commissioned (/ required) by project |
|---|---|---|---|
| External consultant | 27 | 19 | 8 |
| Donor | 5 | 5 | 0 |
| Project | 5 | 2 | 3 |
| Consultant + Donor | 2 | 2 | 0 |
| Consultant + Project | 1 | 0 | 1 |
| Donor + Project | 1 | 1 | 0 |
| Unknown | 6 | 6 | 0 |
| Total | 47 | 35 | 12 |
Figure 1
Timing of Evaluation (no. years following implementation)
Figure 2
Crude Split of Evaluation Reports by Methodology
Table 2
Participatory Approaches in the Sample of Evaluation Reports
| Participatory decision-making in evaluation priorities and methods | Participatory/consultative data collection | Participatory data analysis | Stated purposes for participation | |
|---|---|---|---|---|
| (Jallov & Lwange-Ntale, 2006) | “Evaluation launch meetings” with all relevant stakeholders to “articulate their needs, interests and expectations”. | Not indicated. | “Debriefing meetings” to confront “all relevant stakeholders” with intermediary results to hear and include their reactions. | Encourage ownership, evaluation not as control but as interactive learning process. |
| (Shresta, 2007) | Project managers (does not use the term ‘participation’). | Workshop to collect ‘change stories’ (with reference to the MSC technique). | Vote on the most significant change stories. | Reason not stated. |
| (Thompson, 2006) | Not indicated. | Partner staff (radio/TV stations) (does not use the term ‘participation’). | Not indicated. | Reason not stated, implies efficient data collection method. |
| (Renneberg, Green, Kapera, & Manguy, 2010) | Not indicated. | Partner staff (radio stations) (does not use the term ‘participation’). | Not indicated. | Reasons not stated, implies efficient data collection method. |
| (Jallov & Lwanga-Ntale, 2007) | Not indicated. | Communities involved in collecting change stories. Evaluator consolidated and verified. | Partner staff (radio station) were involved in prioritisation. | Useful when no indicators. Implies that purpose is to reflect local perspectives. |
| (Jallov, 2006) | Not indicated. | Not indicated. | Confronted them with intermediary results to hear and include their reactions. | Encourage ownership, evaluation as interactive learning process. |
| (Taouti-Cherif, 2008) | Project managers (does not use the term ‘participation’). | Not indicated. | Not indicated. | Reason not stated. |
| (Cohen, Zivetz, & Malan, 2008) | Not indicated. | Not indicated. | Allowed staff to comment, no direct say over final report. | Incorporate staff input. |
| (Stiles, 2006) | Program managers. | Not indicated. | Not indicated. | For utilisation approach, participation to focus evaluation on improvement. |
| (Cornell, 2006) | Not indicated. | Not indicated. | Presented initial findings to donors, program staff. Comments were included. | Purpose not given. |
Appendix 1
Sample of Evaluation Reports (in order of publication)
| Year | Author | Title |
|---|---|---|
| 2003 | De Luce | Assessment of USAID media assistance in Bosnia and Herzegovina, 1996–2002 |
| 2003 | Rockwell & Kumar | Journalism training and institution building in Central American countries |
| 2003 | Kumar & Randall Cooper | Promoting independent media in Russia: an assessment of USAID's media assistance |
| 2004 | McClear & Koenig | Mid-term assessment of IREX Media Innovations Program |
| 2004 | ARD Inc. | Montenegro media assessment and evaluation of USAID media interventions: final report |
| 2004 | Lipuscek | ERNO television news project for the Western Balkan region: assessment report for UNESCO-final |
| 2005 | Mytton | Evaluation and Review of Hannu Daya in Jigawa State |
| 2005 | Kalathil & Kumar | USAID's media assistance: strengthening independent radio in Indonesia |
| 2005 | Soloway & Saddigue | USAID's assistance to the media sector in Afghanistan |
| 2006 | Creative Associates International | Haiti media assistance and civic education program (RAMAK). Final report |
| 2006 | Jallov | Journalism as a tool for the formation of a free, informed and participatory democratic development: Swedish support to a Palestinian journalist training project on the West Bank and Gaza for the period 1996–2005 |
| 2006 | Jallov & Lwanga-Ntale | Swedish Support to a Regional Environmental Journalism and Communication Programme in Eastern Africa for the Period 2002–2006 |
| 2006 | Elmqvist & Bastian | Promoting media professionalism, independence and accountability in Sri Lanka |
| 2006 | Intergovernmental Council of the IPDC | Expanding PII Community Feature Network and Grassroots Publication |
| 2006 | Kessler & Faye | INFORMO(T)RAC Programme – Joint Review Mission Report |
| 2006 | Intergovernmental Council of the IPDC | Workshops on low cost digital production systems |
| 2006 | Intergovernmental Council of the IPDC | AIDCOM: Sensitising and Educating the Rural Journalists on Press Freedom and Pluralistic Society |
| 2006 | Intergovernmental Council of the IPDC | Diversifying Information and Improving Radio Programme Production through the Digitalisation of Radio Archives |
| 2006 | Cornell & Thielen | Assessment of USAID/Bosnia and Herzegovina media interventions: final report |
| 2006 | Skjeseth, Hayat & Raphael | Journalists as power brokers: review of the South Asian Free Media Association (SAFMA) and the Free Media Foundation (FMF) |
| 2006 | Thompson | Evaluation report on Medienhilfe network projects in Macedonia and Kosovo |
| 2006 | Sayagues | Writing for Our Lives: How the Maisha Yetu Project Changed Health Coverage in Africa |
| 2006 | Stiles & Weeks | Towards an improved strategy of support to public service broadcasting: evaluation of UNESCO's support to public service broadcasting |
| 2007 | Martinez-Cajas, Invernizzi, Schader, Ntemgwa & Wainberg | The Impact of the J2J Program On Worldwide HIV Awareness |
| 2007 | Shrestha | An evaluation report on “Building ICT opportunities for development communications” project: a part of the Building Communication Opportunities (BCO) programme |
| 2007 | Jallov & Lwanga-Ntale | Impact Assessment of East African Community Media Project 2000 – 2006: Report from Orkonerei Radio Service (ORS) in Tanzania and Selected Communities |
| 2007 | Pradhan | Tracer Study on Training Graduates of Media Centre Programme Panos South Asia |
| 2008 | Unknown | Political Crisis, Mediated Deliberation and Citizen Engagement: A case study of Bangladesh and Nirbachoni Sanglap |
| 2008 | Intergovernmental Council of the IPDC | Palestine: Empowering the Media Sector in Hebron |
| 2008 | Cohen, Zivetz &a Malan | Training Journalists to Report on HIV/AIDS: Final Evaluation of a Global Program |
| 2008 | Taouti-Cherif | Evaluation of Search for Common Ground-Talking Drum Studio Sierra Leone Election Strategy 2007 |
| 2008 | Intergovernmental Council of the IPDC | Creation of a Mayan Communication Network – REFCOMAYA |
| 2008 | Intergovernmental Council of the IPDC | Palestine Studio for Children's Programmes at the Palestinian Broadcasting Corporation (PBC) |
| 2008 | Intergovernmental Council of the IPDC | Palestine: Giving Women a Voice |
| 2008 | Intergovernmental Council of the IPDC | Training Journalists in Freedom of Expression and Indigenous Rights |
| 2008 | Intergovernmental Council of the IPDC | Nepal (various projects) |
| 2009 | SNV Netherlands Development Organisation | Engaging Media in Local Governance Processes: The Case of Radio Sibuka, Shinyanga Press Club, and Kagera Press Club |
| 2009 | Renneberg, Thompson, Taurakoto & Walliker | Independent Evaluation of ‘Vois Blong Yumi’ Program, Vanuatu |
| 2009 | Graham | WFSJ Peer-to-Peer Mentoring Project (SjCOOP): Evaluation and Recommendations |
| 2009 | Anonymous | Final program report: core media support program for Armenia |
| 2010 | Renneberg, Green, Kapera & Manguy | Papua New Guinea Media Development Initiative 2. Evaluation Report. |
| 2011 | ICFJ | An evaluation of the Knight International Journalism Fellowships |
| 2011 | Warnock | Driving Change Through Rural Radio Debate in Uganda |
| 2011 | Internews | Communication in crisis: assessing the impact of Mayardit FM following the May 2011 Abyei emergency |
| 2011 | Development and Training Services | Final report mid-term evaluation: Serbia Media Assessment Program |
| 2011 | Myers | Mid-Term Review BBC World Trust Project ‘A National Conversation’ Funded under DFID's Governance and Transparency Fund |
| 2012 | O’Keefe | Independent Evaluation of PNG Media for Development Initiative: Joint AusAID-NBC-ABC Management Response |
Appendix 2
List of Evaluators Interviewed
| Name | Affiliation | Sector | Nationality | Gender | Date of interview | Communication type |
|---|---|---|---|---|---|---|
| Robyn Patricia Renneberg | Consultant | Development evaluation | Australian | Female | 26/02/2013 | Skype (video) |
| John Cohn | Consultant (once only) | Media assistance evaluation | American | Male | 27/02/2013 | Skype (video) |
| Birgitte Jallov | Consultant | Media assistance (and C4D) evaluation | Danish | Female | 6/03/2013 | Skype (audio) |
| Scott Herrling, MS | Consultant – Philliber Research Associates | General evaluation | American | Male | 13/03/2013 | Skype (audio) |
| Dr Mary Myers | Consultant | Media assistance evaluation | British | Female | 20/03/2013 | Skype (video) |
| Kitty Warnock | Consultant (formerly internal for Panos) | Media assistance evaluation | British | Female | 9/04/2013 | Skype (audio) |
| Tara Susman-Peña | Internews | Internal media assistance research management (not an evaluator) | American | Female | 24/07/2013 | Skype (video) |
| Susan Abbott | Internews (/academic) | Internal media assistance research management | American | Female | 26/07/2013 | Skype (video) |
| Maureen Taylor | Consultant (/academic) (formerly internal at IREX) | Media assistance evaluation | American | Female | 28/08/2013 | Brisbane |
| Adrienne Testa | BBC Media Action | Internal research management | British | Female | 18/09/2013 | Skype (audio) |