Myanmar Baptist Convention

Evaluating their impact

The Myanmar Baptist Convention was formed in 1865 and is the largest Christian organisation in Myanmar. It works with 16 regional language conventions around the country.

The Christian Service and Social Development Department within MBC (known as the CSSDD) has worked in partnership with Tearfund for a number of years on a capacity-building programme. This was planned jointly during 1996 in order to build up the capacity of local leaders. It involved three aspects:

• Leadership training
• Seed funds (start-up funds) for community-based initiatives
• Building up sustainability.

The programme was based around a series of training workshops. The training was targeted at three levels – development directors, development workers and the communities. Each regional convention was invited to send its development director and to select a number of development workers according to the size of the convention. Some conventions, such as the Kachin, Karen, Zomi and Lahu were very enthusiastic, while others did not take the training opportunity seriously and selected only one worker. Several senior management personnel and ten development directors attended.

The programme took place over a period of three years from 1997 under the guidance of Louie Cadaing, a trainer from the Philippines. Louie ran three training workshops covering the above subjects and each training was repeated three times. Each workshop lasted ten days with a six month break between them. Initially 85 trainees began but only 69 trainees completed all three training sessions and shared the teaching within their own language areas. These trainees then had access to seed funds for initiatives within their communities.

From the beginning, evaluation was built into the planning. A baseline study was carried out. Every 3–4 months an internal review took place to provide a quick assessment of how things were going.

Process of evaluation 

Tearfund and CSSDD agreed dates for a final joint evaluation but in fact only one person from Tearfund was able to attend. Instead this time was used by Louie and the CSSDD staff to discuss and plan for the evaluation. This pre-evaluation became a very powerful time. It gave the opportunity for MBC staff to own the evaluation. They planned the extent of the evaluation, drew up the necessary questions and planned teams and venues.

The evaluation took place several months later. It looked at the impact of the capacity-building training by considering how effectively the trainees had shared their training and then facilitated an appropriate project using seed funds. There were three teams of three evaluators including MBC staff, consultants from within Myanmar and Tearfund staff.

There were two days of initial preparation, briefing and training. The evaluation team agreed the following ground rules:

• Freedom to check out ideas and opinions with other team members
• Freedom to allow people to tell their stories
• Confidentiality for participants
• Conclusions must be based on evidence.

Due to excellent advance planning by MBC, the analysis with communities took just three days. One team stayed in Yangon visiting NGOs and MBC leaders. The other two teams visited a total of five communities and six convention leaders. The communities enjoyed the participatory activities including mapping, ranking and polarisation. The openness of Tearfund staff and the other evaluators encouraged everyone else to be open.

The five finger planning method

Check-list for impact evaluations

• Ensure that all the key people are involved in planning the evaluation.
• Plan to evaluate impact from the beginning.
• Encourage the beneficiaries to participate fully. The exercises and friendly atmosphere encouraged people to relax and enjoy the participatory evaluation exercises.
• Be clear about the reasons for evaluating impact. Evaluation was done to discover the impact of the work of the programme on the poor and to enable reflection on whether a change of direction was needed.
• Use the findings. Too often evaluation reports sit on shelves and gather dust! MBC and Tearfund have based all future planning around the findings.

• What is the purpose of the evaluation and who is asking for it?
• How will information be gathered?
• Who will be involved and what resources will they need?
• Where will the work be done?
• When will it take place?

Objectives and indicators

The evaluation included the following objectives:

• What was the impact of the training programme on the participants?
• What was the impact of the seed fund programme?
• How much impact did the whole programme have on the poor?
• How easily could the programme be replicated?
• A number of indicators were agreed for the evaluation such as:
• whether trainers had passed on their training to others
• growth in confidence and skills of trainers
• increase in trainers’ awareness of development issues
• sustainability of projects
• whether projects were meeting needs expressed by the village people
• changes in people’s attitudes and socio-economic situation
• replication of project ideas
• whether projects were tackling the perceived root causes of poverty.

Participatory methods, observation and discussion were combined and used very effectively. For example, villagers were asked what their biggest problems had been during the last three years. They drew a table in the sand and graded the problems using one to ten stones: the more stones, the bigger the problem. They did this for each of the last three years. The results highlighted a lack of knowledge and new techniques, lack of capital, poor communications and support for micro-enterprise. Evaluators then discussed how effectively the project had helped with each of these problems.

The examples were very varied. In one village the community had used the seed funding to bring water to the village via pipes from a water source one mile away. This had been tried years earlier, but without the right materials. The water was now distributed evenly around the whole village and many people had started little kitchen gardens with the extra water available.

However, another project, providing pigs to poor families, had not worked so well. Some of the pigs had died of disease under village conditions. Some families ended up using their own food supplies for the pigs’ food. People were more used to looking after local breeds of pig. There had been little discussion before the project to find out what the village people wanted.

There was a debriefing of one day, followed by analysis of the findings. The content of the evaluation and recommendations were agreed together.

Over half the trainees had shared information from their training. The seed projects had the potential to help the poorest people in their communities in a cost effective way, though some were much more effective than others. Sustainability was a concern. MBC staff commented, ‘Now we realise how important community involvement is’; ‘Participants’ attitudes have changed considerably and they are now open to working with other language groups.’ They appreciated the methods used and the opportunity to reflect on the impact of their work and plan accordingly for the future.

CSSDD, MBC, PO Box 506, Yangon, Myanmar.