This page is part of our successful commissioning toolkit.
Purpose of evaluation
It is important to evaluate policies and programmes. It allows us to learn the lessons from one generation of public service and build them into improvements in the next. [Note]
Approaches to evaluation
There is a range of approaches to evaluation. In general, a public body will commission an independent organisation – perhaps a higher education institution or independent research centre – to carry out major evaluations on its behalf. We will not seek to repeat that professional knowledge here. But we do make a few general points.
The two main approaches to evaluation are summative and formative evaluation.
Summative evaluation, also known as impact evaluation, asks questions about the impact of a specified programme on a specific group of people. This is clearly more straightforward if you have been clear about the outcomes of the programme. Summative evaluation asks how the impact compares to the original objectives, or to some other programme, or to doing nothing at all.
Formative evaluation, also known as process evaluation, asks how and why a programme has worked (or not). A formative evaluation typically studies the development of the policy and its implementation and delivery.
Other questions in the design of an evaluation include:
- The balance in the use of quantitative and qualitative information;
- The extent of use of experimental techniques, such as randomised control.
The role of third sector organisations in evaluation
Third sector organisations (TSOs) have a strong role to play in evaluation. The members of TSOs are often well placed to answer the important questions in evaluations, such as who benefited, and why? In many cases, a TSO will have the capacity and credibility to carry out the evaluation itself.
Practical example: Evaluation
A government department is setting up a programme to open up wider areas of the countryside for leisure use by people from disadvantaged groups. The department decides to carry out a summative evaluation of this programme.
Successful summative evaluation depends on clarity as to the outcomes of the programme. The outcomes of this programme are potentially difficult to discern. Is it about the number of people from disadvantaged groups using the countryside? Is it about their satisfaction with the experience? Is it about longer-term benefits, such as health? Is opening up certain areas ‘worth more’ than others? Is greater social cohesion between countryside residents and the visitors important?
Fortunately, the department was clear about desired outcomes when it was establishing the programme. It therefore can move quickly to set up the summative evaluation. It uses a procurement process for this. The winner is a consortium made up of a higher education institution, a TSO that focuses on countryside issues and a TSO that works on community cohesion.
Notes
Note: National guidance on this area is contained in Government Social Research Unit, The Magenta Book: Guidance Notes for Policy Evaluation and Analysis, HM Treasury, 2007.
Related content
Table of contents
Successful commissioning toolkit: Table of contents