The Impact Of Low-Stakes Accountability On Teachers Work
Author(s):
Esteban Rozenwajn (presenting / submitting)
Conference:
ECER 2014
Format:
Paper

Session Information

23 SES 06 C, Accountability, (In)equality and Social Justice

Paper Session

Time:
2014-09-03
15:30-17:00
Room:
B334 Sala de Aulas
Chair:
Xavier Rambla

Contribution

Standardized testing is not new in western educational systems. But with the rise of accountability policies in the late 80s, external evaluation seems to have conquered the enthusiasm of policy-makers world-wide. Under the imperative of school improvement, new school accountability measures will progressively set performance indicators in the shape of test results as a guiding reference, not only for pupils, but for every level of the educational system.

However, the weight of the social, historical and political context must not be neglected while considering this trend in educational policies. The form accountability devices adopt appears to be very heterogeneous depending on the country or region where it develops. Though, for the sake of simplicity, this great diversity is often reduced to the distinction between high-stakes accountability, generally referring to cases where test results are associated with heavy consequences for educational agents in terms of career or schooling; and low-stakes accountability, referring to the opposite cases where results are not associated to heavy consequences for teachers.

In each of these cases, the rationale concerning the role and the effects of external test on local agents is different (Maroy & Voisin, 2013). High-stakes accountability generally conceives individual behavior in the light of a strategic actor, whose motivation to change behavior is based on the desire to avoid sanction or to earn reward. On the other side, low-stakes accountability models rely on a more reflexive view of human behavior where results should allow individuals to have a better knowledge of the strengths and weaknesses of their own practices. But independently of the accountability type, the basic idea is that results of standardized tests would be able to trigger behaviors that, ultimately, should lead to educational improvement. Beyond those principles, the theoretical framework allowing the explanation of the effects produced by external evaluation on local agents remains greatly under-developed.

A certain number of empirical studies have tried to grasp these changes, enlightening both intended and unintended practices as effects of accountability measures (i.e. Amrein-beardsley, Berliner, & Rideau, 2010; Booher-Jennings, 2005; Haney, 2000; Shepard, 1990). But the relation between external evaluation and teacher practices is not always evident. Sometimes, especially in low-stakes cases, teachers’ even deny their work is affected by accountability measures (Hellrung & Hartig, 2013). Thus, some scholars urged for taking teachers’ perceptions of accountability measures into account (i.e. Dierendonck & Fagnant, 2010; Schildkamp & Kuiper, 2010; Verhaege, Vanhoof, Valcke, & Van Petegem, 2010). Embedded in this line of research, the present study investigates the impact of low-stakes external evaluation on teachers’ work in the context of French-speaking Belgian primary schools. Thus, our main objective is to analyze how teachers perceive and legitimize external evaluation, the way they make sense of test results, and the link between these processes and their practices.

Method

30 semi-directed interviews with teachers working in 10 French-speaking Belgian primary schools were made. According to the main objective of this study, the goal of these interviews was to investigate the perceptions each individual teacher held towards external evaluation and the way they reacted to it. In particular, we looked at (1) how teachers perceived and legitimized external evaluation, (2) the way they made sense of test results and (3) the way their practices had changed as a consequence of external evaluation. Schools were selected as to be diverse in the socioeconomic level of the public they hosted. In each school, several teachers from different grade levels were then freely invited to participate in the study. This way, school social dynamics impact on teachers’ perceptions of external evaluation could be investigated. Moreover, French-speaking Belgian educational system is provided with different accountability devices. Formative external evaluation is imposed to 3rd and 5th grade level pupils, while a summative external evaluation grants pupils with a diploma leading to secondary education. Thus, having teachers from different grade levels allowed to analyze the impact of distinct accountability devices. Collected data from interviews was then qualitatively analyzed with the help of NVivo 10 software. Inductive and deductive coding oriented our attention towards dimensions of acceptance/legitimization (basic principle acceptance, function acceptance, test acceptance), results processing (responsibility, causes of performance) and practices (consequences of external evaluation on pedagogical practices and non-pedagogical practices). Links between perceptions and sense-making processes were then explored.

Expected Outcomes

First of all, data analysis shows that teachers globally accept the idea of external testing while implementation is greatly criticized. But above all, teachers seem to reject the validity of results as a personal performance feedback, thus rejecting personal responsibility in test results. External factors as socioeconomical background of students were most often invoked to explain test results. However, external evaluation appeared to be a cause of frustration for many teachers. A significant part of teachers from low SES schools expressed a feeling of being held accountable to political and administrative structures for test results, which caused great anxiety for them. Teachers from higher SES schools for their part, often felt discomfort with external test results as the high success rate of their pupils may hinder parent demands for excellency. These accountability feelings were accentuated for late summative external evaluation in contrast to early formative external evaluation. Independently of teacher perceptions and feelings however, test results’ impact on teachers’ practices showed to be limited and unexpected. Teachers’ practices were indeed found to be adapted to external evaluation contents in a way that could be labeled as teaching to the test: using test items for instruction and intense preparation before testing were often revealed. These results show that despite official political discourse assuring that low-stakes accountability measures would not be used for judging their work, teachers often perceived them as if they were. However, teachers’ feelings of being held accountable did not lead them making instructional changes in accordance to test results as “reflexive” theories of the external evaluation would have suggested. As it is, external evaluation, independently of test results, appeared to be used by teachers as a reference guideline for their instructional practices, indicating that mandatory external evaluation is a sufficient condition to trigger teaching to the test.

References

Amrein-beardsley, A., Berliner, D. C., & Rideau, S. (2010). Cheating in the first, second and third degree: Educators’ responses to high-stakes testing. Educacion Policy Analysis Archives, 18(14), 1–36. Au, W. (2007). High-Stakes Testing and Curricular Control: A Qualitative Metasynthesis. Educational Researcher, 36(5), 258–267. Booher-Jennings, J. (2005). Below the bubble:“Educational triage” and the Texas Accountability System. American Educational Research Journal, 42(2), 231–268. Clarke, M., Shore, A., Rhoades, K., Abrams, L., Miao, J., & Li, J. (2003). Percieved effects of state-mandated testing programs on educators in low-, medium-, and high stakes states. Chestnut Hill, MA. Diamond, J. B. (2007). Where the rubber meets the road: Rethinking the connection between high-stakes testing policy and classroom instruction. Sociology of Education2, 80(4), 285–313. Dierendonck, C., & Fagnant, A. (2010). Quelques réflexions autour des épreuves d’évaluation développées dans le cadre de l'approche par compétences. Le Bulletin de l’ADMEE-EUROPE, 1. Firestone, W. A., Mayrowetz, D., & Fairman, J. (1998). Performance-based assessment and instructional change: The effects of testing in Maine and Maryland. Educational Evaluation and Policy Analysis, 20(2), 95–113. Haney, W. (2000). The Myth of the Texas Miracle in Education. Education Policy Analysis Archives, 8(41). Hellrung, K., & Hartig, J. (2013). Understanding and using feedback – A review of empirical studies concerning feedback from external evaluations to teachers. Educational Research Review, 9, 174–190. Linn, R. L. (2000). Assessments and accountability. Educational Researcher, 29(2), 4–16. Maroy, C., & Voisin, A. (2013). Une typologie des politiques d’accountability en éducation: l'incidence de l'instrumentalisation et des théories de la régulation. Mons, N. (2009). Les effets théoriques et réels de l’évaluation standardisée. Eurydice. Shepard, L. A. (1990). Inflated test score gains: Is the problem old norms or teaching the test? Educational Measurement: Issues and Practice, 9(3), 15–22. Schildkamp, K., & Kuiper, W. (2010). Data-informed curriculum reform: Which data, what purposes, and promoting and hindering factors. Teaching and Teacher Education, 26(3), 482–496. Smith, L. M. (1991). Put to the test: The effects of external testing on teachers. Educational Researcher, 20 (5), 8-11. Spillane, J. P. (2012). Data in Practice : Conceptualizing the Data-Based Decision-Making Phenomena. American Journal of Education, 118(2), 113–141. Verhaege, G., Vanhoof, J., Valcke, M., & Van Petegem, P. (2010). Using school performance feedback : Perceptions of primary school principals about performance feedback use. School Effectiveness and School Improvement, 37(2), 141–154. Visscher, A. J., & Coe, R. (2003). School performance feedback systems : conceptualisation, analysis and reflection. School Effectiveness and School Improvement, 14(3), 321–349.

Author Information

Esteban Rozenwajn (presenting / submitting)
Université Catholique de Louvain (UCL)
Pshychology
Brussels

Update Modus of this Database

The current conference programme can be browsed in the conference management system (conftool) and, closer to the conference, in the conference app.
This database will be updated with the conference data after ECER. 

Search the ECER Programme

  • Search for keywords and phrases in "Text Search"
  • Restrict in which part of the abstracts to search in "Where to search"
  • Search for authors and in the respective field.
  • For planning your conference attendance, please use the conference app, which will be issued some weeks before the conference and the conference agenda provided in conftool.
  • If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.