Assessing Teachers’ Learning Trajectory Oriented Formative Assessment Practices in Mathematics

Author(s):

Jonathan Supovitz(presenting / submitting)

Conference:

ECER 2014

Network:

24. Mathematics Education Research

Format:

Paper

Session Information

24 SES 06 A, Mathematics Teacher Professional Development

Paper Session

Time:

2014-09-03

15:30-17:00

Room:

B113 Sala de Aulas

Chair:

Birgit Pepin

Contribution

The current Standards reform movement in mathematics in the United States calls for increasing the rigor of teachers’ ability to recognize the sophistication of student understanding, placing this in a developmental context, and taking appropriate instructional steps in response. The Consortium for Policy Research in Education (CPRE) at the University of Pennsylvania has developed a series of grade specific, on-line, mathematics teacher assessments that measure these instructional capabilities in mathematics.The instrument, called the TASK, or Teacher Analysis of Student Knowledge, measures important components of the instructional knowledge necessary to teach to the high expectations of the Standards. In this paper we the TASK instrument and a study of its administration with 1,400 mathematics teachers in grades kindergarten through 10.

A TASK is an on-line teacher assessment that presents a teacher with a carefully designed set of student responses to a mathematics problem. The student responses characterize different levels of sophistication of student thinking as well as common misconceptions. Prompts ask the teacher to examine students’ solution strategies, to analyze students’ thinking, and to provide subsequent instructional suggestions. (Ebby, Sirinides, Supovitz, & Oettinger, 2013)

The conceptual framework of the TASK is based on the concept of learning trajectory-oriented formative assessment. Formative assessment is considered one of the most promising methods of facilitating student learning. The basis of any potentially formative experience is both a clear understanding of the gap between a learner’s current state and the goal of learning, or standard, and the pathway to achieve the goal. A well-designed assessment helps to locate the learner on the pathway towards the goal. The assessment becomes formative when the information it contains provides feedback to either the learner and/or the teacher. For the instructional feedback loop to close, the teacher then has to provide an informed instructional response to the learner that helps move them closer to the goal. Knowledge of the learning trajectory helps the teacher both locate the learner on the pathway and develop specific hypotheses about what kinds of assistance will help the learner move towards the goal. Formative assessment is an iterative process, so the cycle repeats until the gap between the learner’s current state and the goal is closed and/or new learning goals are established (Ramaprasad, 1983; Sadler, 1989)

A synthesis of the research on formative assessment by Black and Wiliam (1998) founds substantial evidence of large gains in student learning when teachers employ formative assessment practices. More recently, researchers have begun to piece together the ways in which students progress as they develop mathematical understanding, called learning trajectories (Daro, Mosher, Corcoran, 2011).

This paper and presentation will describe the instrument and a field trial conducted with 1200 teachers of mathematics in the United States. More specifically, the work will explore the following research questions:

What does performance look like within each of the six domains assessed by the TASK instrument?
What are some of the capabilities and weakness of teacher performance across the six domains?
What teacher and organizational characteristics are related to TASK performance?

Method

In the Spring of 2012, CPRE conducted a large field trial in partnership with five public school districts in five northeastern and southern states in the United States. The districts varied in size, student demographics, and programs of mathematics instruction. Altogether, the five school districts contained 417 schools and served about 230,000 students. To achieve our final teacher sample, we randomly drew 1,851 teachers from the five districts. Of these, 1,386 responded, for a 75% response rate. The instrument, administered on-line, is then scored by trained scorers who produce scores for each of the six domains of the instrument. This paper and presentation will report the descriptive statistics for teacher responses at different grade levels in each of the six domains measured by the instrument. The report also includes the results of a series of regression analyses that examine the relationship between teacher professional characteristics (teaching experience, professional development, degree level) beliefs (teaching orientation, beliefs about mathematics, confidence teaching mathematics), and TASK performance.

Expected Outcomes

These findings reflect a general picture of the current state of teachers’ learning trajectory-oriented formative assessment capabilities in grades K-10 in five urban and urban fringe districts in five American states. The findings suggest that across the domains examined on the TASK, there were more procedural responses than any other category. In fact, with the exception of Analysis of Student Thinking in fractions, the procedural responses outnumbered conceptual and learning trajectory responses combined. Given the emphasis in the Standards on rigor as a balance between conceptual and procedural understanding, this suggests that there is a great deal of room for growth in teacher capacity to identify, interpret, and respond to students’ conceptual understanding. The findings also show that, although about 40% of teachers are able to identify the mathematics concept that an item intends to assess, when examining the student work of that item, more than three quarters of the teachers focused on what students do (procedural emphasis) rather than what they understand (conceptual emphasis). Teachers were more successful at ranking student work in order of sophistication than they were in providing a reasoned rationale for that ranking. This suggests that while many teachers may have developed a tacit sense of levels of sophistication in student reasoning, their knowledge about the developmental nature of student thinking is not well articulated. Additionally, the results indicate that teaching experience was negatively correlated with TASK performance, suggesting younger teachers actually had higher performance. Teachers’ beliefs were not correlated with performance, although performance was positively associated with teachers’ sense of confidence as mathematics teachers.

References

Black, P., & Wiliam, D. (1998). Assessment and classroom learning.Assessment in education, 5(1), 7-74. Clements, D. H., & Sarama, J. (2004). Learning trajectories in mathematics education. Mathematical Thinking and Learning, 6(2), 81-89. Daro, P., Mosher, F. A., & Corcoran, T. (2011). Learning trajectories in mathematics (Vol. 68). Research Report. Philadelphia, PA: Consortium for Policy Research in Education Ebby, C. B., Sirinides, P.; Supovitz, J.A.; Oettinger, A. (2013) Teacher Analysis of Student Knowledge (TASK): Technical Report. Philadelphia, PA: Consortium for Policy Research in Education Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological bulletin, 119(2), 254. Ramaprasad, A. (1983). On the definition of feedback. Behavioral Science,28(1), 4-13. Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional science, 18(2), 119-144.

Author Information

Jonathan Supovitz (presenting / submitting)

University of Pennsylvania

Graduate School of Education

Philadelphia

Search the ECER Programme

Search for keywords and phrases in "Text Search"
Restrict in which part of the abstracts to search in "Where to search"
Search for authors and in the respective field.
For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.