Footsteps magazine issues on a wooden desk.

From: Footsteps 50

How to understand the impact we are having and adjust our work accordingly

Evaluating their impact

The Myanmar Baptist Convention was formed in 1865 and is the largest Christian organisation in Myanmar. It works with 16 regional language conventions around the country.

The Christian Service and Social Development Department within MBC (known as the CSSDD) has worked in partnership with Tearfund for a number of years on a capacity-building programme. This was planned jointly during 1996 in order to build up the capacity of local leaders. It involved three aspects:

The programme was based around a series of training workshops. The training was targeted at three levels – development directors, development workers and the communities. Each regional convention was invited to send its development director and to select a number of development workers according to the size of the convention. Some conventions, such as the Kachin, Karen, Zomi and Lahu were very enthusiastic, while others did not take the training opportunity seriously and selected only one worker. Several senior management personnel and ten development directors attended.

The programme took place over a period of three years from 1997 under the guidance of Louie Cadaing, a trainer from the Philippines. Louie ran three training workshops covering the above subjects and each training was repeated three times. Each workshop lasted ten days with a six month break between them. Initially 85 trainees began but only 69 trainees completed all three training sessions and shared the teaching within their own language areas. These trainees then had access to seed funds for initiatives within their communities.

From the beginning, evaluation was built into the planning. A baseline study was carried out. Every 3–4 months an internal review took place to provide a quick assessment of how things were going.

Process of evaluation 

Tearfund and CSSDD agreed dates for a final joint evaluation but in fact only one person from Tearfund was able to attend. Instead this time was used by Louie and the CSSDD staff to discuss and plan for the evaluation. This pre-evaluation became a very powerful time. It gave the opportunity for MBC staff to own the evaluation. They planned the extent of the evaluation, drew up the necessary questions and planned teams and venues.

The evaluation took place several months later. It looked at the impact of the capacity-building training by considering how effectively the trainees had shared their training and then facilitated an appropriate project using seed funds. There were three teams of three evaluators including MBC staff, consultants from within Myanmar and Tearfund staff.

There were two days of initial preparation, briefing and training. The evaluation team agreed the following ground rules:

Due to excellent advance planning by MBC, the analysis with communities took just three days. One team stayed in Yangon visiting NGOs and MBC leaders. The other two teams visited a total of five communities and six convention leaders. The communities enjoyed the participatory activities including mapping, ranking and polarisation. The openness of Tearfund staff and the other evaluators encouraged everyone else to be open.

The five finger planning method

 

Check-list for impact evaluations

Objectives and indicators

The evaluation included the following objectives:

Participatory methods, observation and discussion were combined and used very effectively. For example, villagers were asked what their biggest problems had been during the last three years. They drew a table in the sand and graded the problems using one to ten stones: the more stones, the bigger the problem. They did this for each of the last three years. The results highlighted a lack of knowledge and new techniques, lack of capital, poor communications and support for micro-enterprise. Evaluators then discussed how effectively the project had helped with each of these problems.

The examples were very varied. In one village the community had used the seed funding to bring water to the village via pipes from a water source one mile away. This had been tried years earlier, but without the right materials. The water was now distributed evenly around the whole village and many people had started little kitchen gardens with the extra water available.

However, another project, providing pigs to poor families, had not worked so well. Some of the pigs had died of disease under village conditions. Some families ended up using their own food supplies for the pigs’ food. People were more used to looking after local breeds of pig. There had been little discussion before the project to find out what the village people wanted.

There was a debriefing of one day, followed by analysis of the findings. The content of the evaluation and recommendations were agreed together.

Over half the trainees had shared information from their training. The seed projects had the potential to help the poorest people in their communities in a cost effective way, though some were much more effective than others. Sustainability was a concern. MBC staff commented, ‘Now we realise how important community involvement is’; ‘Participants’ attitudes have changed considerably and they are now open to working with other language groups.’ They appreciated the methods used and the opportunity to reflect on the impact of their work and plan accordingly for the future.

CSSDD, MBC, PO Box 506, Yangon, Myanmar.

Similarly Tagged Content

Share this resource

If you found this resource useful, please share it with others so they can benefit too.