PLAIN TEXT - How evaluators can support community-led evaluation

Authored by Laura Fisher.

Published August 2021.

Community groups often find it hard to set aside time to evaluate the work they do in their local area. There can also be times when research or evaluation requests from others do not match with what communities themselves would like to understand about their work. 

This paper explores different approaches to community-level evaluation and looks at how to ensure they are useful and engaging for communities.

Local Trust has piloted support that aims to improve the monitoring, evaluation and impact measurement skills of volunteer-led community groups. This paper draws on the findings from the pilot of the ​‘measuring change’ support and outlines the approaches evaluators and researchers can take to support community-led evaluation. Local Trust’s evaluation of the pilot of ​‘measuring change’ found that community groups have three main needs that enable them to effectively evaluate their work. They would like to:

• improve their skills and capacities
• improve their strategic thinking
• ensure that they feel able to continue evaluation in the future. 

Co-produced data collection tools and frameworks can help to develop capacity for communities to understand the changes they make in their area, as long as they do not require too much additional capacity from community groups going forward.

Finally, this paper highlights how evaluators and researchers should allow significant time and space for volunteers to understand their own needs around evaluation and how it can be relevant and useful to the work they do in their communities. Providing flexible support and ensuring space for reflective practice are also key approaches in ensuring volunteer buy-in and that community groups are able to get the most out of the process in terms of skill and capacity development. 

Studies show that evaluation generally can be a complex process and ​“communities often lack the resources, skills, time and budget” to measure and evaluate their own work (Dunkley and Franklin, 2017, p. 114). Acknowledging these challenges, Local Trust has offered support to Big Local areas to better measure the changes they are making in their communities since 2019.

Big Local is a resident-led funding programme run by Local Trust providing people in 150 areas in England with £1.15m each to spend across 10–15 years to create lasting change in their neighbourhoods. The measuring change support sits alongside a broader range of support available for Big Local areas to help them to deliver Big Local. The name of the support reflects the resident-led ethos of the programme and dispenses of terms such as ​‘impact’ and ​‘evaluation’ as those are often regarded as things that funders do to communities, rather than something that a community has ownership of.

The support began in 2019 as a result of growing interest from Big Local areas which has risen dramatically over the past few years as areas make plans for their legacy. Initial priorities of the Big Local areas usually did not include evaluation as their focus was on understanding and delivering the programme.

The measuring change support offer aims to develop these skills by matching Big Local partnerships with specialist providers to work at a timeframe that suits the group’s needs. This helps to overcome the resource and time challenges communities face when it comes to evaluating their work. Local Trust also covers the cost of the support, so budget is not an issue.

The purpose of this offer is to provide a community-led approach to evaluation in order to improve residents’ skills and experience in monitoring, evaluation and impact measurement. Since its inception, 16 Big Local partnerships have been paired with community-led evaluation specialists to work together to understand the changes they make in their area. A Big Local partnership is a group made up of at least eight people that guides the overall direction of a Big Local area.

This support is demand-led and resident-led, which means the support can take a variety of forms. Local Trust works with the Big Local partnership to understand how best they can be supported and if the support is right for them, before the partnership is able to select the provider they would like to work with. Together the volunteers and provider may work together to create an evaluation plan to understand the differences they are making in their community, undertake a community research project with residents or evaluate specific projects run in the Big Local area.

The outcomes of the measuring change support are to: 

• Increase residents’ skills and capacity to measure the change they are making.
• Inform Local Trust on the approaches that are most effective in supporting Big Local areas to measure the change they are making.
• Collect more rigorous data and research, for use by both Big Local areas and Local Trust, on the changes they are bringing about.
• Support Big Local areas to better work with external organisations to evaluate their activities. 

With resources and time being scarce for community groups interested in evaluating their work, the additional capacity provided through direct support for communities is invaluable. This section identifies the main support needs of communities who want to measure change in their area, as identified through Local Trust’s pilot support offer:

• improve their skills and capacities
• improve their strategic thinking
• ensure evaluation continues beyond the support. 

A key need of communities wanting to measure the changes they make in their area is the development of knowledge, skills and capacity to be able to undertake monitoring, research or evaluation. Following the measuring change support, Big Local partnerships reported a greater understanding of impact measurement and felt they now understood both what is measurable and how to measure it. There has also been a change in how some volunteers perceive evaluation, with a recognition that evaluation is not just something ​“that big funders do” but is also achievable and appropriate for their work too.

Confidence in partnerships also increased, with chairs of Big Local partnerships reporting how partnership members now feel they have the confidence to speak about how their work contributes to bigger aims in the community, or how to request appropriate monitoring information from external organisations that they fund. They feel this will help them better question the projects that they do and understand how individual projects or activities fit into a wider whole, as well as help them to better hold others to account, such as external delivery partners.

Alongside a better understanding of measurement, partnerships have also developed a greater awareness of the processes that sit alongside a successful evaluation framework, including data collection, data storage and the need to start considering project evaluation from project conception. At the start of the measuring change support, many Big Local partnerships identified how they had previously gathered a lot of data but had done very little with it. Now partnership members feel they understand why they collect data and what they can do with it. They see the benefits of having a bank of useful data, analysis and learning to draw on in the future.

While the evaluation of the measuring change support showed that, initially, learning and development resides with a select few within the Big Local partnerships, volunteers are keen to embed and expand this learning to those who were not directly involved in the support. This may be challenging at first, however they see the evidence gathered about the strength of their work as an important way of building the interest, and then skills, of others volunteering in their community. 

The links between evaluation and better planning strategically are clear, but for some of the Big Local areas involved in the measuring change pilot, the development of this skill has been a positive, but unintended consequence. Over the period of work with their provider, partnerships began to recognise the links between the support and how they can demonstrate the legacy they will leave behind in their local community. Better strategic thinking and processes to enable forward planning are a real benefit to those working to create change in their local communities, however they may not always be recognised as an immediate need in the face of more short-term motivations for evaluation.

Through the creation of evaluation frameworks and theories of change, partnership members have unlocked ​‘key questions’ they can ask themselves about the work they deliver, commission or contract. Chairs of Big Local partnerships involved in the support reported better strategic thinking from partnership members who now had a greater awareness of how projects and activities should fit in with their broader aims for their community. They have reported how the creation of clear frameworks has allowed for more nuance in their decision-making and that identifying gaps in delivering or looking beyond ​‘pet projects’ would now be possible. 

As community groups can often feel they lack the capacity to undertake evaluation of their work, it is important that they feel able to continue to do this after the end of any formal support they receive. This is one of the main objectives of the measuring change support for Big Local areas and it is clear that where volunteers have worked with providers to develop evaluation frameworks or tools this has better enabled partnerships to see that evaluation is part of a process that will extend beyond the lifetime of the support. Big Local partnerships now feel they understand what they need to do to measure the changes they make in their communities in the future, but they also now have the tools to be able to do this in a way that does not place a greater burden on their limited capacity.

Big Local partnerships recognise that embedding their new processes and frameworks will help ensure that all partnership members understand the value of measuring change in their community and see this as something that will strengthen over time. One Big Local partnership is attempting to demonstrate the value of new processes and their new framework to all partnership members by nominating project ​“champions” which will encourage board members to use their tools and processes to have oversight of their chosen projects. 

Through the measuring change support pilot, Local Trust identified some key ways evaluators and researchers can support areas to meet the needs of communities interested in understanding the changes they make in their area. These are:

• supporting the community to identify their own needs
• facilitated space for reflection
• flexibility.

For volunteers to be engaged with the measuring change support, it was incredibly important that they understood what their own needs were for evaluating the work that they do. During the measuring change support, this worked well when the Big Local partnership had a specific project in mind that they wanted to evaluate or a question they wanted to explore about their work or their community. However not all partnerships knew what they wanted to use the support for. While they knew that impact measurement was important, they had not yet identified a clear need for how the support would fit into their work.

During the measuring change support, what worked well was when providers were able to take an aspect of a partnership’s activities or plans and show how the measuring change support could help with understanding the impact that it has, improving its delivery, or promoting its benefits to the community. However, it is important that evaluators and researchers give communities time and space to identify their own support needs, and this may involve allowing communities to come to you when they have identified what their evaluation need is. This need may also sit separately from any funder requirements for evaluation. Building in time for communities to understand their evaluation needs may take longer than anticipated or change over time, particularly for volunteers who are new to the concept of evaluation, but it is the best way to keep volunteered engaged in the process from the outset. 

The need for additional capacity to support volunteer groups with evaluation feels obvious, but this extends beyond the requirement of bringing in an organisation or individual with evaluation knowledge. A chair of a Big Local partnership spoke of the benefits of having an external provider who was seen as a neutral figure lead workshops with volunteers on the partnership. Through the measuring change support, Big Local partnerships have been able to take part in facilitated reflective sessions that have led to a greater understanding of their impact and how they can measure this in the future. Skilled facilitators meant that volunteers were able to push through some of the more challenging aspects of evaluation, such as conversations about overall aims and whether all projects contribute to these outcomes. Having this space for reflection enabled volunteers to recognise the changes they may need to make to meet their ambitions for their community. 

The flexibility of the measuring change support model has been key to maintaining momentum and partnership interest in the work over time as circumstances change within local communities. This flexibility is achieved in multiple aspects of the support. As projects in Big Local areas changed, support providers had to adjust existing workplans to the new context. This was even more important due to the COVID-19 pandemic. Frequent communication and a willingness to adjust plans to meet new contexts was cited by Big Local partnerships as an essential feature of their relationship with their support provider.

While the support aims to be resident-led, there is still space for providers to suggest how their time could be used during evaluation support. For those who are particularly new to the idea of evaluation, it can be too much pressure to ask volunteer groups to develop a workplan on their own, particularly for long-term support across several months. During the measuring change support, where providers were able to guide or suggest ways forward for partnerships, project momentum was maintained, even if workplans had to be adapted as time went on.

There was also a need for providers to be flexible in who they work with during the support. Big Local partnership members are volunteers who had varying levels of involvement based on their own personal interests and capacity. Providers needed to ensure they were able to provide opportunities for different levels of engagement for volunteers, such as working closely with subgroups to progress the support, running final ​‘wrap-up’ workshops for all partnership members or providing frequent agenda items at partnership meetings to share progress with the whole group. 

This briefing has taken the findings from an evaluation of Local Trust’s measuring change support for Big Local areas to detail some of the main needs of communities wanting to measure change in their area, as well as ways evaluation specialists can best support them.

Additional capacity in the form of expert support will always be a benefit to community-led groups who may face constraints of budget, time, knowledge and resources. Communities and volunteers have clear needs around improving their skills and capacities and improving their strategic thinking, both of which can be developed through direct support. By ensuring the buy-in of volunteers in the community and working with them to develop tools and frameworks, evaluation support providers can ensure that communities feel able to continue to measure changes in their area long after the support has ended.

The pilot evaluation from the measuring change support identified three main ways that evaluators can meet the support needs of communities who are interested in understanding the changes that they make in their area. Allowing the community the time and space to identify their own need for evaluation and offering flexible support and reflective practice also all contribute towards volunteer buy-in both during and after the support and contribute to a stronger understanding of both the concept and process of evaluation.

Taking on board learning from the pilot around the importance of flexibility and the time needed for volunteer groups to identify their evaluation needs, Local Trust have introduced a more flexible support offer that allows partnerships to take up a shorter portion of support that may fit better with their context and expectations. One area has already completed the first phase of this flexible support offer and has decided to move on to a further phase of support with their support provider. This suggests that the more flexible model allows for a natural progression through different types of impact measurement support, rather than expecting an up-front, longer-term commitment from volunteers which could feel intimidating for some. Local Trust will be continuing to learn from the measuring change support and improving our understanding of community-led evaluation as more Big Local areas engage in the support over the next year. 

Dunkley, R.A. and Franklin, A. ​‘Failing better: The stochastic art of evaluating community-led environmental programs’. Evaluation and Program Planning, 60, pp. 112–122. (2017). 

Local Trust (2020), ​‘Measuring Change.’ [No longer available].

Local Trust (2021), ​‘Measuring change support pilot evaluation.’ Available at: https://www.learningfrombiglocal.org.uk/resources/measuring-change-support-pilot-evaluation.