10 Years of Evaluation Practice in Media Assistance: Who, When, Why and How?

Noske-Turner, Jessica

doi:10.1515/nor-2015-0028

Figures & Tables

Timing of Evaluation (no. years following implementation)

Crude Split of Evaluation Reports by Methodology

Sample of Evaluation Reports (in order of publication)

Year	Author	Title
2003	De Luce	Assessment of USAID media assistance in Bosnia and Herzegovina, 1996–2002
2003	Rockwell & Kumar	Journalism training and institution building in Central American countries
2003	Kumar & Randall Cooper	Promoting independent media in Russia: an assessment of USAID's media assistance
2004	McClear & Koenig	Mid-term assessment of IREX Media Innovations Program
2004	ARD Inc.	Montenegro media assessment and evaluation of USAID media interventions: final report
2004	Lipuscek	ERNO television news project for the Western Balkan region: assessment report for UNESCO-final
2005	Mytton	Evaluation and Review of Hannu Daya in Jigawa State
2005	Kalathil & Kumar	USAID's media assistance: strengthening independent radio in Indonesia
2005	Soloway & Saddigue	USAID's assistance to the media sector in Afghanistan
2006	Creative Associates International	Haiti media assistance and civic education program (RAMAK). Final report
2006	Jallov	Journalism as a tool for the formation of a free, informed and participatory democratic development: Swedish support to a Palestinian journalist training project on the West Bank and Gaza for the period 1996–2005
2006	Jallov & Lwanga-Ntale	Swedish Support to a Regional Environmental Journalism and Communication Programme in Eastern Africa for the Period 2002–2006
2006	Elmqvist & Bastian	Promoting media professionalism, independence and accountability in Sri Lanka
2006	Intergovernmental Council of the IPDC	Expanding PII Community Feature Network and Grassroots Publication
2006	Kessler & Faye	INFORMO(T)RAC Programme – Joint Review Mission Report
2006	Intergovernmental Council of the IPDC	Workshops on low cost digital production systems
2006	Intergovernmental Council of the IPDC	AIDCOM: Sensitising and Educating the Rural Journalists on Press Freedom and Pluralistic Society
2006	Intergovernmental Council of the IPDC	Diversifying Information and Improving Radio Programme Production through the Digitalisation of Radio Archives
2006	Cornell & Thielen	Assessment of USAID/Bosnia and Herzegovina media interventions: final report
2006	Skjeseth, Hayat & Raphael	Journalists as power brokers: review of the South Asian Free Media Association (SAFMA) and the Free Media Foundation (FMF)
2006	Thompson	Evaluation report on Medienhilfe network projects in Macedonia and Kosovo
2006	Sayagues	Writing for Our Lives: How the Maisha Yetu Project Changed Health Coverage in Africa
2006	Stiles & Weeks	Towards an improved strategy of support to public service broadcasting: evaluation of UNESCO's support to public service broadcasting
2007	Martinez-Cajas, Invernizzi, Schader, Ntemgwa & Wainberg	The Impact of the J2J Program On Worldwide HIV Awareness
2007	Shrestha	An evaluation report on “Building ICT opportunities for development communications” project: a part of the Building Communication Opportunities (BCO) programme
2007	Jallov & Lwanga-Ntale	Impact Assessment of East African Community Media Project 2000 – 2006: Report from Orkonerei Radio Service (ORS) in Tanzania and Selected Communities
2007	Pradhan	Tracer Study on Training Graduates of Media Centre Programme Panos South Asia
2008	Unknown	Political Crisis, Mediated Deliberation and Citizen Engagement: A case study of Bangladesh and Nirbachoni Sanglap
2008	Intergovernmental Council of the IPDC	Palestine: Empowering the Media Sector in Hebron
2008	Cohen, Zivetz &a Malan	Training Journalists to Report on HIV/AIDS: Final Evaluation of a Global Program
2008	Taouti-Cherif	Evaluation of Search for Common Ground-Talking Drum Studio Sierra Leone Election Strategy 2007
2008	Intergovernmental Council of the IPDC	Creation of a Mayan Communication Network – REFCOMAYA
2008	Intergovernmental Council of the IPDC	Palestine Studio for Children's Programmes at the Palestinian Broadcasting Corporation (PBC)
2008	Intergovernmental Council of the IPDC	Palestine: Giving Women a Voice
2008	Intergovernmental Council of the IPDC	Training Journalists in Freedom of Expression and Indigenous Rights
2008	Intergovernmental Council of the IPDC	Nepal (various projects)
2009	SNV Netherlands Development Organisation	Engaging Media in Local Governance Processes: The Case of Radio Sibuka, Shinyanga Press Club, and Kagera Press Club
2009	Renneberg, Thompson, Taurakoto & Walliker	Independent Evaluation of ‘Vois Blong Yumi’ Program, Vanuatu
2009	Graham	WFSJ Peer-to-Peer Mentoring Project (SjCOOP): Evaluation and Recommendations
2009	Anonymous	Final program report: core media support program for Armenia
2010	Renneberg, Green, Kapera & Manguy	Papua New Guinea Media Development Initiative 2. Evaluation Report.
2011	ICFJ	An evaluation of the Knight International Journalism Fellowships
2011	Warnock	Driving Change Through Rural Radio Debate in Uganda
2011	Internews	Communication in crisis: assessing the impact of Mayardit FM following the May 2011 Abyei emergency
2011	Development and Training Services	Final report mid-term evaluation: Serbia Media Assessment Program
2011	Myers	Mid-Term Review BBC World Trust Project ‘A National Conversation’ Funded under DFID's Governance and Transparency Fund
2012	O’Keefe	Independent Evaluation of PNG Media for Development Initiative: Joint AusAID-NBC-ABC Management Response

List of Evaluators Interviewed

Name	Affiliation	Sector	Nationality	Gender	Date of interview	Communication type
Robyn Patricia Renneberg	Consultant	Development evaluation	Australian	Female	26/02/2013	Skype (video)
John Cohn	Consultant (once only)	Media assistance evaluation	American	Male	27/02/2013	Skype (video)
Birgitte Jallov	Consultant	Media assistance (and C4D) evaluation	Danish	Female	6/03/2013	Skype (audio)
Scott Herrling, MS	Consultant – Philliber Research Associates	General evaluation	American	Male	13/03/2013	Skype (audio)
Dr Mary Myers	Consultant	Media assistance evaluation	British	Female	20/03/2013	Skype (video)
Kitty Warnock	Consultant (formerly internal for Panos)	Media assistance evaluation	British	Female	9/04/2013	Skype (audio)
Tara Susman-Peña	Internews	Internal media assistance research management (not an evaluator)	American	Female	24/07/2013	Skype (video)
Susan Abbott	Internews (/academic)	Internal media assistance research management	American	Female	26/07/2013	Skype (video)
Maureen Taylor	Consultant (/academic) (formerly internal at IREX)	Media assistance evaluation	American	Female	28/08/2013	Brisbane
Adrienne Testa	BBC Media Action	Internal research management	British	Female	18/09/2013	Skype (audio)

Authorship of the Sample of Media Assistance Evaluation Reports

Authored by	Total	Commissioned (/ required) by donor	Commissioned (/ required) by project
External consultant	27	19	8
Donor	5	5	0
Project	5	2	3
Consultant + Donor	2	2	0
Consultant + Project	1	0	1
Donor + Project	1	1	0
Unknown	6	6	0
Total	47	35	12

Participatory Approaches in the Sample of Evaluation Reports

	Participatory decision-making in evaluation priorities and methods	Participatory/consultative data collection	Participatory data analysis	Stated purposes for participation
(Jallov & Lwange-Ntale, 2006)	“Evaluation launch meetings” with all relevant stakeholders to “articulate their needs, interests and expectations”.	Not indicated.	“Debriefing meetings” to confront “all relevant stakeholders” with intermediary results to hear and include their reactions.	Encourage ownership, evaluation not as control but as interactive learning process.
(Shresta, 2007)	Project managers (does not use the term ‘participation’).	Workshop to collect ‘change stories’ (with reference to the MSC technique).	Vote on the most significant change stories.	Reason not stated.
(Thompson, 2006)	Not indicated.	Partner staff (radio/TV stations) (does not use the term ‘participation’).	Not indicated.	Reason not stated, implies efficient data collection method.
(Renneberg, Green, Kapera, & Manguy, 2010)	Not indicated.	Partner staff (radio stations) (does not use the term ‘participation’).	Not indicated.	Reasons not stated, implies efficient data collection method.
(Jallov & Lwanga-Ntale, 2007)	Not indicated.	Communities involved in collecting change stories. Evaluator consolidated and verified.	Partner staff (radio station) were involved in prioritisation.	Useful when no indicators. Implies that purpose is to reflect local perspectives.
(Jallov, 2006)	Not indicated.	Not indicated.	Confronted them with intermediary results to hear and include their reactions.	Encourage ownership, evaluation as interactive learning process.
(Taouti-Cherif, 2008)	Project managers (does not use the term ‘participation’).	Not indicated.	Not indicated.	Reason not stated.
(Cohen, Zivetz, & Malan, 2008)	Not indicated.	Not indicated.	Allowed staff to comment, no direct say over final report.	Incorporate staff input.
(Stiles, 2006)	Program managers.	Not indicated.	Not indicated.	For utilisation approach, participation to focus evaluation on improvement.
(Cornell, 2006)	Not indicated.	Not indicated.	Presented initial findings to donors, program staff. Comments were included.	Purpose not given.

10 Years of Evaluation Practice in Media Assistance: Who, When, Why and How?

Figures & Tables

Figure 1

Figure 2

Sample of Evaluation Reports (in order of publication)

List of Evaluators Interviewed

Authorship of the Sample of Media Assistance Evaluation Reports

Participatory Approaches in the Sample of Evaluation Reports

Paradigm

My account