City Research Online

Abstract

This research investigated ways in which the impact of EU and similarly funded projects can be evaluated beyond the immediate conclusion of the project work. The research is focused on the use of Contribution Analysis (CA) to prepare a Theory of Change (ToC) and to determine whether and to what extent projects may be considered to have made a meaningful contribution.

The potential of using this approach to evaluate future projects is also considered in this research. The research also utilised Bayesian Confidence Updating and Pawson’s Categories of Contextual Influences methods, which were applied to two of the Tempus project objectives. These methods were incorporated into the research to test if they could help strengthen the contribution narrative for each objective, which proved successful and added rigour to the findings. Both methods helped enhance the project’s contribution story and demonstrate that additional methods can help strengthen CA. A comprehensive explanation of how both methods were applied to the research is provided in Appendix K for Bayesian Confidence Updating and Appendix Y for Pawson’s Categories of Contextual Influences.

The projects that are the subject of this study are the EU-funded TEMPUS project Developing Information Literacy for Lifelong Learning and Knowledge Economy in Western Balkan Countries, which ran from 2011 to 2015, and the Erasmus+ Capacity Building in the Field of Higher Education (CBHE) funded Library Network Support Services (LNSS) project which was active between the years 2015-2018.

The research question to be addressed was, “Can Contribution Analysis (a Theory Based approach to evaluation) be used to evaluate the long-term impact of EU Erasmus+ projects effectively?”

The research had set out to address two sub-research questions:

(1) Measuring the effects and impacts of these LIS projects can help demonstrate the viability and worth of investments in EU Erasmus+ projects in the Western Balkans region. And:
(2) Demonstrating the impact of these projects in a more visual, coherent, and systematic way will help realise the benefits of such projects for individuals and regions.

Having performed CA on the two projects and assessed all the objectives individually, it is reasonable to assume, based on the evidence assembled and the empirical evidence gathered, that

the project actions were responsible in the majority of cases, for the detected results that in essence, the objectives were delivered by the projects. While other influencing factors were mentioned and considered, in most cases, these influences were marginal. In both the questionnaires and the interviews, participants who took part in this research attested that the project activities had medium and longer-term impacts. This research statement would require additional investigation. However, the evidence discovered during the empirical evidence-gathering phase would suggest that the projects had both direct and indirect influences (Montague et al., 2002, as cited in Mayne, 2008).

The research findings would strongly suggest that CA could be used successfully to evaluate the long-term impact of projects. There are caveats, however, regarding the need for the rigor of evaluation theory on the one hand and the need to tailor to the needs of the intervention to improve the evaluation theory on the other and perhaps even include other methods to help reinforce the contribution narrative.

This research also highlighted the limitations of the current EACEA system of project evaluation, that most of the evaluation is done within the project life cycle, and the evidence of the impact of the project in the medium and longer term is incomplete.

In the case of the EACEA and similar projects, it would be my recommendation that the 6-steps of CA (Mayne, 2008) be followed, and a rigorous ToC established. CA could be woven into the current evaluation system employed by the EACEA and be a condition for example on funding being awarded, as per the framework suggested in this research.

Evaluation of projects is important to the EACEA, and measuring the effect of projects has been ongoing since these EU funding programmes began. Measuring impact cannot just be a mantra, and every effort should be made to evaluate each project’s contribution. CA can help the funding agency understand how and why the intervention has contributed and give the Agency the information required to demonstrate this to its stakeholders (the Citizens of Europe).

Publication Type:	Thesis (Doctoral)
Subjects:	J Political Science > JN Political institutions (Europe) L Education > LB Theory and practice of education > LB2300 Higher Education Z Bibliography. Library Science. Information Resources > Z665 Library Science. Information Science
Departments:	School of Communication & Creativity > Media, Culture & Creative Industries > Library & Information Science School of Communication & Creativity > School of Communication & Creativity Doctoral Theses Doctoral Theses

Export

Downloads