“You can’t put children and young people in petri dishes,” is not a phrase you hear every day. But it’s a striking image, that captures the challenge of how to effectively evaluate the impact of our work with children and young people.
Delivering and measuring positive outcomes for children and young people was the subject of our first Roundtable discussion event on Monday 27 November. We were delighted to be joined by a wide range of partners, commissioners and sector colleagues from across Sussex and Surrey representing all the areas in which we work, including housing, mental health, specialist advice and youth work.
The discussion was led by Professor Kaz Stuart, from the Centre for Youth Impact, who gave a thought-provoking presentation on the history and evolution of evaluation, what evaluation and evidence really are, and what current, best and future practices might look like. (Slides available to download HERE).
Breakout sessions produced interesting discussions. Common themes included frustration that evaluation is tied directly to funding, measuring outputs rather than outcomes, a focus on short-term results rather than long-term change and the trend towards using medical models.
“If you can’t prove it works, you can’t have the money.”
“Often evaluation is led by funders, but it should be the other way around. We are the experts in the outcomes and should lead on how to evidence it.”
“Evaluation feels imposed on projects – it feels like it is bolted on and doesn’t fit the work we’re doing.”
“The medical model of measurement is the bane of my life. It has no bearing on real life with young people. How they change and develop isn’t a linear path.”
“I’m concerned about the increasing use of ‘control groups’ as a way of measuring outcomes. It can corrupt the process and/or be unethical.”
Another common frustration was the methodology for collecting evidence. Many talked about ‘survey fatigue’ amongst young people, particularly those in minority groups such as LGBTQ+, or care leavers, who are disproportionately asked for feedback. Many felt that it led to young people disengaging with the service.
“It isn’t person-centred to continually ask someone to ‘rate’ the service or hand them an evaluation form every time they’ve taken part in an activity. It can make them feel like they’re just a number.”
Whilst it was clear that there is no easy answer, what came across was a shared aspiration to improve outcomes for children and young people and work more collectively to find ways to improve evaluation and demonstrate impact. There was consensus around focusing on quality to deliver meaningful outcomes and in using participative and co-created evaluation methods.
The second half of the morning focused on how, and why, develop a ‘theory of change’. Using a case study, it explored how a ‘theory of change’ can ensure a project, service, or organisation can clearly articulate how it will deliver quality outcomes.
YMCA DownsLink Group has recently been through the process and El Clarke, Head of Quality Compliance, shared insights, learnings and ‘top tips’ on how best to do it. (Slides available to download HERE). The discussions that followed highlighted the different stages organisations are at. Whilst some are already using one, many are part-way through, and others were inspired to start the process.
“It is interesting to see how a theory of change isn’t just a repeat of our strategy. Before the session I hadn’t really understood the difference. It’s made me see the value of it and I now want to develop our own.”
Emily Brock, CEO YMCA DownsLink Group, ended the session acknowledging how powerful it was that the room was full of stakeholders, who share the aim of driving better outcomes for children and young people. She encouraged everyone to write pledges, one for themselves and one offering to help someone else, to hold each other to account, but also to continue the supportive collaboration fostered during the session.
Emily reflected on Kaz’s presentation and how the phrase, ‘slow down to speed up’ resonated with her and how we might respond to it. Finally, she recognised the value in coming together as a ‘collective voice’ which might have to be prepared to ‘be disruptive’ to drive change and ensure that children and young people, across our region, are getting the high-quality services they deserve.
For more information or to join our mailing list for future sector events email [email protected]
