Terms of Reference Final Evaluation of the EU-co-funded project in Armenia “Access to Information and Investigative Journalism for Better Informed Citizens”

Sourse: http://www.foi.am

Background

 

In the course of the period May 2018 – December 2019 Freedom of Information Center of Armenia (FOICA) has been implementing “Access to Information and Investigative Journalism for Better Informed Citizens” (“Informed Armenia”) project, 90% funded by the European Union and co-funded by Freedom of Information Center of Armenia and European Journalism Centre.

 

The overall objective of the project has been to reinforce civil society in Armenia; in particular, CSOs capable to conduct effective monitoring activity on state budget, as well as young members of civil society capable to conduct journalistic investigations.

 

Key Indicators: 

 

a) 90 national and regional CSOs enhanced with monitoring capacities in area of state budget monitoring, 
b) 10 cases of transparent state budget process development relating to sectors covered by EU-Armenia cooperation, 
c) Younger generation of the Armenian civil society (about 100 final-year law/journalism students) is trained in practice how to investigate and present to the public urgent and topical issues.

 

The specific objectives:

 

1. Watchdog capacity of CSOs, particularly the oversight of the State Budget in sectors relevant for EU/Armenia relations is strengthened.
2. Investigative reporting on state budget and on various human rights issues is increased among the youth.

 

Indicators:

 

1. The recommendations presented (by all the 23 sub-grantees) in their papers are acted upon by the government (at least 50%);
2.The majority of the participating students (of which 50% are expected to be women) envisage to continue their future career as investigative journalists.

 

1st component: Strengthening monitoring capacities of CSOs. 
Activity 1. Trainings were implemented throughout Armenia with the purpose of enabling CSOs (especially youth initiatives and civic groups) and media organizations to gain better understanding of EU-Armenia relations, reforms required by it and access to information issues, international best practices in respect of policies and watchdog strategies relating to the promotion of EU integration and to strengthen their ability to engage in sustained dialogue with authorities.

 

Activity 2. Overall 23 sub-grants were provided to CSOs and media organizations, on a competitive basis, which have completed the training, to undertake monitoring, watchdog activities, as well as for general institutional support. The aim of each sub-grant was to help CSOs to provide assistance to the Government agencies on efficient and effective state budget expenditure on relevant sector reforms required by the EU-Armenia relations, as well as to support CSOs’ institutional capacities for their core mandate advocacy.

 

2nd component: Organizing competition on investigative journalism.
The main activity was to conduct an investigative competition-TV show, the idea of which was to engage final-year journalism (and law) students in competition with each other about who will investigate a topical issue better.
During the preparation period the 36 students for each season were selected, which are supposed to create 12 teams, with 3 members in each team. The Board of the Project developed the assignment-topics for the teams. 
Prizes were given to the 1st, 2nd and 3rd journalistic teams, as well as the winners of the Moot Court. The whole working process of the participants was recorded in reality show format, and was broadcast via contracted web media outlet and also via Public Television of Armenia.

 

Evaluation Methodology

 

It is planned to hire a short-term independent external evaluator to evaluate the progress of the project/action and provide recommendations for similar future projects as well as for some of the weaknesses that may be identified.
The final evaluation of the project will include examining effectiveness of all activities as follows:
-evaluation of the training component
-evaluation of the 1st season of the Journalistic battles competition 
-evaluation of the 2nd season of the Journalistic battles competition 
-evaluation of the specific advocacy grants component (1st round of sub-grants)
-evaluation of the core institutional grants component (2nd round of sub-grants)

 

The Evaluation expert is expected to utilize both secondary and primary research tools and to engage both direct and end beneficiaries. The desk review (of project documentation and other relevant materials) will be supplemented by the fieldwork. Interviews will be conducted with the direct beneficiaries of the project; i.e. participants training courses, sub-grantees, etc. In-depth qualitative interviews will be conducted with the EUD focal point for the project, the project coordinator and managers of the sub-granting projects, as well as wider stakeholders. The survey should critically examine the information gathered from various sources, and synthesize the information in an objective manner.
The evaluation results will be validated with the EUD, national partners and key stakeholders. All collected data needs to be gender-aggregated. Prior to the fieldwork, the evaluator will be responsible for pre-testing the data collection tools to be utilized.

 

After this first part of the assignment, the Evaluator should submit an interim report including a detailed fieldwork plan, the list of people to be interviewed, data collection tools to be used, dates of visit and itinerary. The fieldwork should start once the Interim report is approved.

 

The evaluator should keep close contact with and report to the evaluation manager. 
Upon completing the fieldwork and analysing the data and information the Evaluation expert will draft the final report incorporating the findings, conclusions and recommendations. The draft report will be reviewed by the EUD and Project director for comments and remarks. The Evaluation expert should address and incorporate all these comments and submit the Final Evaluation report.

 

Standard evaluation criteria such as the relevance, effectiveness, efficiency, sustainability and impact (the latter to the extent possible) will be analysed. The evaluation questions will include but are not limited to:

 

Relevance: 
• To what extent were the project objective’s aligned to Armenia's reform priorities and country strategic programs?
• To what extent were the project objective’s aligned to EU-Armenia’s priority areas and strategic programs?
• To what extent are the needs of project target groups met?
• Has the project ensured due commitment and ownership of all the key stakeholders? 
• To what extent did the project supplement and/or contribute to other on-going (and/or planned) projects in the field? 
• To what extent were the project-related data available? Did they suffice to duly define indicators and measure their achievements vs. the project objectives?
• Has the project team taken into consideration the crosscutting issues, such as gender equality of participants? If imbalances, how could they be addressed in the future?

 

Effectiveness:
• What are the beneficiaries’ views about the project components and services received? How effective were the capacity-building activities of beneficiaries? 
• What are the beneficiaries’ views about improving services? Is the quality of outputs satisfactory?
• What are the stakeholders’ views about project components and services provided? Do they spot weaknesses and issues? How they think these issues may be addressed?
• To what extent did the project outputs/activities lead to the expected outcomes (results); shortcomings, achievements? 
• Were the expected results timely and realistically defined and mentioned in the project reports?
• Was the interaction/communication among the key stakeholders and between the key stakeholders and beneficiaries effective to ensure the smooth implementation of the project?

 

Efficiency:
• Was there a clear distribution of roles and responsibilities between the Lead organization and the partner?
• Were the capacities of the Lead organization and the partner sufficient to duly carry out the project?
• Did the project resources correspond to the needs of the action?
• What about the resources from other stakeholders? Were they available? Were they timely? 
• Delays – changed circumstances:
• Was the intervention logic of the project responsive to changed circumstances (delays, capacity of beneficiaries, etc.)?
• Cost-effectiveness:
• Have the outputs been produced/delivered in a cost-efficient manner?
• Could the same results and impacts be achieved in a more cost-effective manner?

 

Sustainability:
• To what extent have the project interventions contributed to the institutional development for the local NGOs to ensure ownership and sustainability of results after the completion of the project? Human and institutional capacity? Financial capacity?
• Is there any support from the public and/or private sector to ensure the sustainability of project outcomes?
• Are there any constraints to the sustainability of results?
• What is the level of ownership of the project outputs among the project beneficiaries? How successful they use those outputs? 
• Did the project adhere to the EU Communication and Visibility guidelines? What are the best communication tools and media engagement lessons learnt from this project? What could be improved?

 

Reporting Requirements

The Evaluator will ensure the provision of all the deliverables as mentioned below. The deadlines and English language quality should be respected and if needed, the language should be improved before the Report submission.

 

• The Evaluator is expected to submit the Interim Report (up to 10 pages) by listing the main evaluation questions and the methodology and timetable of the evaluation, including the data collection tools and number of respondents. 
• The draft report (up to 35 pages excluding annexes) will preliminary answer the evaluation questions and come up with findings being drawn by the secondary review and primary fieldwork. The lists for bibliography, respondents and evaluation questions should be annexed. A summary of the evaluation findings should be included, including two pages detailing the impact of the project as well as key success-stories. 
• The draft Final Report will further consolidate the answers to evaluation questions, synthesize findings, come up with conclusions and propose recommendations towards designing better projects in the field.
• The Final Evaluation Report will duly incorporate any comments received from the EUD and implementing organizations on the draft final report.

 

The Evaluator is expected to keep regular communication with and report to Ms. Shushan Doydoyan, the project director by duly sharing and discussing the issues raised during the evaluation.

 

Indicative planning

Activity	Timetable (2019-2020)
Submission of ToR for project evaluation to the EU Delegation to Armenia by the FOICA	28 August, 2019
Receiving the comments/remarks by the EUD and finalizing the ToR by the FOICA	22 October, 2019
FOICA announcing a call for Evaluator	25 October, 2019
Evaluation started	15 November, 2019
Submission of Inception report by Evaluator	25 November, 2019
Initial findings results drafted	27 December, 2019
Draft internal submission of the Report, receiving comments by the FOICA and its finalization	10 January, 2020
Submission of the draft Final Report to the EUD by FOICA	27 January, 2020
Receiving the EUD comments/remarks on the Report, addressing them and submitting the Final Evaluation Report	7 February, 2020

 

 

Required qualifications

• Master's degree or equivalent academic qualifications in sociology, social sciences, business and economic studies or other related fields
• At least 5 years of experience in evaluation, experience in the ENP region and especially in Armenia would be considered as an asset
• Experience in M&E of capacity development programmes in the areas of civil society, community development, communications/media, policy monitoring/advocacy
• Experience of qualitative research methods and tools
• Ability to analyze data, recommend appropriate actions and prepare reporting documents
• Knowledge of EU programs and policies
• Excellent English writing skills (the Applicant is expected to submit 2 excerpts from his/her previous writing pieces, preferably from evaluation reports and/or surveys), knowledge of Armenian will be considered as a distinctive asset
• PC literacy, excellent writing and communication skills.

 

Application procedures

 

The potential applicant should submit his/her:
• CV showing the relevant experience and education
• Motivation letter showing the rationale for evaluating the given project
• 2 previous writing samples of about 2-3 pages each and
• A brief work plan of 2 pages including the Report outline, activity timetable and total budget.

 

The deadline for submitting the applications is November 8, 2019.
Please send your applications to [email protected] or [email protected] email addresses.