Black and Wiliam’s landmark meta-analysis (1998) ignited worldwide interest in the important role of formative assessment in improving teaching and learning. By adopting formative assessment practices, teachers and schools can enhance student learning, particularly for low achieving students, improve equity in educational outcomes, school attendance and retention (OECD, 2005). Heeding available theory and research, policy makers in a number of countries are making formative assessment a primary approach to educational reform (OECD, 2005); even in the United States with its traditionally strong orientation toward accountability testing, states are increasingly encouraging formative assessment (CCSSO, 2008). The measurement, curriculum development and teacher professional development communities have become increasingly involved. Rationales for the enterprise have been mounted from diverse theoretical perspectives, including cognitivist; psycho-, socio-cultural and/or situational constructivism; motivation and social learning theories, even behaviorist views, in attempts to promote diverse learning goals, such as academic knowledge, metacognitive skills, and motivation for lifelong learning (e.g., Shepard, 2005 for review of theory base). This wealth of perspectives generates rich possibilities, but at the same time may mitigate against a comprehensive approach to improving practice. For example, two themes appear dominant in theory and research, the one emphasizing measurement tools needed to support formative practice and the other emphasizing the process of assessment use, but rarely are the two brought together. Practical tips to help practitioners implement either the tools or the process represent a third common area for advice. The current paper intends to bridge this current divide by proposing a general model defining critical elements in quality, learning-based formative assessment systems (see also DiRanna, Osmundson, Topps, Barakos, Gearheart, Cerwin, Carnahan, Strang, 2008; Herman, Osmundson, Ayala, Schneider, and Timms, 2006). Model elements include learning goals, assessment tools (measures), interpretation and use of results as key components of assessment practice that benefits learning. We illustrate the model and its learning-based measures and processes through several mathematics and science projects currently at the national Center for Research on Evaluation, Standards, and Study Testing (CRESST). An ontology of expected learning goals, in cognitive, affective and/or metacognitive domains is the starting point for both assessment design and use, consistent with syntheses of recent advances in psychometric and cognitive theories (Pelligrino et al, 2001). For example, CRESST’s POWERSOURCE formative assessment program is based in an ontology of key mathematical concepts whose understanding to success in Algebra, the gateway course to college in the United States and the assessments themselves build on cognitive principles for supporting learning and transfer (see Baker, 2007). The primary focus of the paper is a recently completed study of the implementation and effects of a curriculum-embedded, formative assessment system in middle school science. Drawing on assessments representing the learning progression within the curriculum and encompassing both formal, individual assessments and coordinated, “on-the fly” whole class and small group assessment opportunities developed by Rich Shavelson, Mark Wilson and colleagues (Kennedy et al., 2005; Shavelson et al, 2005), the study examines in-depth two components of the proposed model: teachers’ interpretation of students’ learning and their use of results in feedback and planning for subsequent teaching and learning. Research questions include: • To what extent and how do teachers use assessment to improve student learning? • How accurate are teachers’ judgments of their students learning? • To what extent does teachers’ accuracy and use of assessment influence student learning?

Methodology The study combined qualitative and quantitative methodologies in analyzing data from eight middle school classrooms that had implemented a unit from the Foundational Approaches in Science Teaching (FAST) curriculum (Pottenger & Young, 1992). Learning-based, the unit’s specially developed formative assessments involved students in a sequence of “predict, observe, and explain” (POE) activities at three time points during the curriculum. The assessments can be viewed as a coordinated system, in that, as noted above, they involved whole class and small group instructional activities designed to provide opportunities for “on-the-fly” assessment as well individual work that could be scored for conceptual understanding. Teacher manuals laid out the expected structure and sequence for the assessments and provided sample discussion questions, guidance on interpreting and deepening student understanding, and advice on next steps for students evidencing various misconceptions. Study measures included student performance on the reflective lessons, pre-and post assessments and a range of observation, interview and log data on teachers’ interpretation and use of the reflective lessons. A common progress variable was used to score all student measures (Kennedy et al, 2005; Wilson & Sloane, 2000), yielding the opportunity to examine trajectories of student learning over five time points. Descriptive and correlation statistics as well as advanced HM models were used to address study questions.

Conclusions Study results showed that teachers made limited use of the available assessment data and experienced difficulty in using whole class and small group discussion to probe and extend student understanding. Analyses of the consistency between teachers’ judgments of student understanding and centralized scores of student work similarly showed uneven results. Interestingly, however, while the study found little relationship between teachers’ use of assessment and student performance, there was a strong positive relationship between the accuracy of teachers’ judgments and student learning. The paper explores study implications for the power and challenges of implementing learning-based assessment systems and recommendations for research, policy and practice.