IAs mentioned before, the evaluators of a program may often belong to the program population itself. The extent of participation of the stakeholders in an evaluation may vary from minimal (where the stakeholder is only a passive respondent to queries asked by the evaluator) to moderate (where the stakeholder takes part either in design or in execution of the evaluation) and further, to maximum (where the stakeholder is actively involved at each stage – identification of the research issues, planning of the evaluation, execution, processing of research data, and dissemination of evaluation findings).
Active participation of common people in an impact evaluation signals their strong sense of ownership of the program and a deep concern to know and control its long-term outcomes.
Participatory impact evaluation may not differ much from a conventional, third party evaluation since, except for the evaluator and the measurement tool, all other elements of evaluation are fairly similar in both cases. In participatory evaluation, the evaluator may belong to the community of immediate stakeholders. Secondly, the tool of measurement may be a simplified one that can be easily designed and administered by the community evaluators as well as easily understood by the respondents.
Justification for Participatory Impact Evaluation (PIE)
As compared to a conventional impact evaluation, the PIE can add value to the evaluation process in the following ways. It can inform about:
- Whether the program has achieved its intended goal as understood by the people (and not those proclaimed by the program management)
- What sort of information was collected and processed and what tools were prepared to measure the impact (thus, making the stakeholders aware of the “complete picture”; as against this, in a conventional evaluation, all that a stakeholder might get to see is the final evaluation report)
- What, according to the people, were the actual factors that caused a change (a community would logically have a much better understanding of the “forces at work” in its area as against an external agency)
Challenges and constraints to Participatory Impact Evaluation
In the context of the use of impact evaluation as a tool of participatory research, the following factors may act as important constraints and can make the evaluation results questionable:
- Since in participatory research, the evaluators belong to the stakeholder group, they might tend to sample the respondents from groups/ households known/ friendly/ relatively more accessible to them. This would introduce sampling errors and affect the final outcome of the study.
- In participatory research conducted by community-based organizations that favor a particular ideology, e.g. socialism or communism, the final analyses may be accordingly biased.
- Evaluators selected from the general public are often not equipped with a complete understanding of statistical concepts, questionnaire design skills, project management skills and methods of analysis of findings, as compared to the professionals that belong to third party evaluation agencies.
Tools used for Participatory Impact Evaluation
The commonly used tools of participatory evaluation include Participatory Rural Appraisal/Rapid Rural Appraisal Toolkits. These tools help understand the impact from the community’s point of view and that. Some of the key tools used include:
- Resource Mapping: Resource maps prepared by the community in pre and post project scenario can vividly bring out changes in the physical and natural resource base as a result of a particular project.
- Wealth Ranking: This tool can capture the impact of a project on the relative income and asset ownership levels in the community.
- Time Line: Time line can be used to capture trends in important socio-economic change points happening as an result of natural calamities, displacement or any economic crisis.
Results from these participatory appraisals can be fed back into the project learning cycle to make the project more community friendly and pro-poor.
