In this paper we examine critically the uses to which models for evaluating CPD activities are put, utilising an examination of established models in the literature, alongside an appraisal of the use we have made of our own model, which draws from a number of these. Our approach to evaluation is influenced by Pawson and Tilley’s (1997) model of Realist/ic Evaluation. Drawing on their discussion of three basic types of evaluation (Experimentalist, Realist, Constructivist) we assess a range of previous CPD evaluation models including some that have influenced ours (Guskey, 2000; Kirkpatrick, 1983; Leithwood and Levin 2005) or been offered by others (such as Goodall et al 2005; Harland and Kinder, 1997). We note the varying uses that such models are put to including heuristic devices; frames for statistical modelling; and as a way of structuring data gathering and interpretation. The main body of the paper discusses the use of one such model, which we have developed, in practice, and which draws on Pawson and Tilley’s (1997) ideas. They posit that effective evaluation requires an understanding of the mechanisms through which particular activities are expected to lead to particular outcomes and the ways in which particular contexts mediate the expected effects. We argue that that the effects of CPD programmes can be modelled using a number of types of variables: • Interventions: the CPD activities themselves; • Antecedents: those factors associated with individual participants that affect their ability to benefit from the opportunities offered to them; and moderating factors: variables in the school and wider environment that influence whether, and how, the interventions lead via the achievement of intermediate outcomes to produce final outcomes. These factors help to explain why apparently similar activities have different consequences for different individuals, teams and schools; • Intermediate outcomes: those outcomes of the CPD activities that are conceived to be pre-conditions for the achievement of the final outcomes, particularly individual learning, changes in behaviour and engagement in particular tasks or activities; and final outcomes: the intended effects of the CPD activities, primarily relating to effects on organisations, teachers and pupils. These variables interact in often complex ways which are sensitive to the details of design and implementation of particular CPD activities.

The paper evaluates the use of the model in a series of multiple method studies we have undertaken for England’s National College for School Leadership (NCSL) and, latterly, the Training and Development Agency for Schools (TDA), including: • Evaluations of individual programmes, such as an evaluation of Leadership Pathways, an NCSL programme for experienced school middle leaders. • Comparisons between programmes, such as our examination of the impact of in-school elements of three other NCSL leadership development programmes (Simkins et al 2009). • Studies of complex programme with multiple aims, recipients and forms, such as the 14-19 Leadership and Management Development programme to support implementation of the new diplomas (Maxwell and Coldwell, 2008) • CPD strategy analysis, exemplified by our evaluation of the Testbed Whole School Training and Development project for the TDA (Coldwell et al, 2008).

We conclude by examining what our experiences tell us about the potentially positive uses to which such models can be put, as well as their limitations. In doing so we compare the different kinds of studies of CPD interventions that evaluators may be asked to undertake, from ‘classic’ evaluations of individual programmes with clearly specified target groups and outcomes, through studies of complex suites of programmes with varying designs and purposes, to more generic policy frameworks which are designed to facilitate professional development in ways which may not be captured by traditional ideas of formal programmes. In our analysis, the problems associated with applying single models to complex and differing CPD contexts is made clear. Finally we address the implications of the complexity of the issue for policy-makers who wish to support CPD evaluation.