The Rest of the Story: Teachers’ Use of Self-narrative to Construct Their Identities as Participants in a Teacher Evaluation System

Author(s):

Noelle Paufler(presenting / submitting)

Conference:

ECER 2016

Network:

09. Assessment, Evaluation, Testing and Measurement

Format:

Paper

Session Information

09 SES 12 B, Formative and Summative Assessments

Paper Session

Time:

2016-08-26

09:00-10:30

Room:

NM-F103a

Chair:

Josef Basl

Contribution

Purpose

In the context of increasing national and global interest in teacher accountability systems that rely at least in part on measures of student achievement, school districts in the United States, much like their counterparts in other countries, have been prompted to implement policy-directed, although often locally-developed, teacher evaluation systems (American Educational Research Association (AERA), 2015; American Statistical Association (ASA), 2014; Amrein-Beardsley, 2014; Braun, 2005; Collins & Amrein-Beardsley, 2014; Harris, 2011). Such systems are based on the premise that teacher quality can be both defined as a construct and quantified for the expressed purpose of making inferences about the effectiveness of individual teachers as professionals (Weisberg, Sexton, Mulhern, & Keeling, 2009). The construct of teacher quality, its purportedly quantifiable domains, and the inherent inferences generated from its reification in policy and practice must, accordingly, be better understood in terms of their policy contexts.

The purpose of this study was to better understand the perceptions of teachers regarding the purpose, components, implementation, and impact of a new policy-directed teacher evaluation system. To do this, the researcher interviewed elementary and high school teachers in a large suburban school district in the United States and utilized self-narrative as a framework for understanding the “possible lives” teachers construct when telling their stories.

Perspectives

The need to recognize and understand the cultural embeddedness of current educational policies from a global perspective is paramount. Self-narrative as the construction of a longitudinal version of self through storytelling can be aptly applied to contemporary issues in educational policy (Bruner, 1991, 2002), including teacher accountability systems and the impact of such systems on practitioners in context. The cultural and linguistic determinants of self-narrative processes can wield power of their own by structuring experiences and organizing memories—in essence, one’s life becomes the narrative told (Bruner, 2004). In the application of narrative as a framework for teacher evaluation, the researcher recognized the fundamental importance of the culturally- and linguistically-bound “possible lives” from which teachers can choose when telling their self-narratives (Bruner 2004).

As discussed here, the “possible lives” reflect the “policy lives” in which the teachers are enveloped, specifically given the culture-laden and linguistically-bounded definitions under which they must operate when demonstrating their own effectiveness. In this case, teachers across the state were evaluated annually based on locally-developed systems mandated to include measures of professional practice (e.g., most often classroom observations) and student achievement (e.g., based on complex statistical value-added models (VAMs)). The preliminary findings in this study suggest that teachers construct self-narratives bounded in part by the policy-directed effectiveness label they receive as part of the evalution process (i.e., highly effective, effective, developing, ineffective). Teachers rely on these labels when choosing from “possible lives” that focus on their understanding of themselves as a good teacher. Understanding these “policy lives” was a preeminent focus in this study.

Method

Methods and Analyses The researcher interviewed 12 elementary and high school teachers to better understand how they use storytelling to construct their own identities as teacher evaluation system participants. The researcher analyzed interview data and identified a series of “policy lives” based on the most prominent self-narratives told by teachers. The Highly Effective Teacher “Highly effective” teachers generally told stories that relied at least in part on evaluation system components as evidence of their professional quality, often in terms of the characteristics and behaviors delineated in the observation rubric or their students’ achievement (e.g., to some extent evidenced by their scores on large-scaled standardized tests). Their stories attributed value to (or at least did not detract from the value of) the evaluation process. Even when not formally articulated, these teachers inherently legitimized their effectiveness label by constructing a self-narrative bounded by system components. The Effective Teacher Although “effective” teachers also told stories structured at least in part based on the evaluation system, their stories often extended beyond the characteristics cited in the rubric as traits of a good teacher and rarely included students’ test scores as primary evidence, generally devaluing the statistical measure of student achievement. The effective teachers’ self-narratives also reflected the boundedness of their “policy lives” in their utilization and omission of evaluation system components. The Developing Teacher Perhaps unsurprisingly, “developing” often told stories that challenged the evaluation system in some way and generally constructed their self-narrative based on a belief that they were in fact better teachers than might be inferred from their effectiveness label. They questioned the use of an arguably incomplete observation rubric or the validity, reliability, and/or fairness of the statistical model used to estimate their impact on student achievement via test scores. Developing teachers’ overall reliance upon the evaluation system as a structure further reflects the boundedness of “policy lives” in their use of narrative as a tool. The Ineffective Teacher “Ineffective” teachers generally cited their less than favorable evaluation outcomes as counter intuitive based on their understandings of self or even irrelevant given the system’s arguable inadequacies. If they acknowledged areas for professional growth, they often disregarded their evaluation outcome as evidence of such. Others cited a detrimental effect on their professional sense of self. In this respect, the evaluation system not only provided a structure for teachers’ construction of self, its adverse impact on their narrative actually reinforced the structure.

Expected Outcomes

Conclusions Although the self-narratives presented herein are arguably simplistic, they provide foundational descriptions of “policy lives” that merit more careful consideration and greater depth of analyses. Understanding the perceptions of teachers in this study through the “policy lives” they construct and how these perceptions might be anticipated elsewhere if/when such policies are adopted is critical given teachers’ marginalization from policy discussions in many cases prior to development and implementation of these evaluation systems. Presumably intended to prompt teacher reflection, especially among those deemed ineffective, the prominence of the evaluation system in teachers’ narratives may further legitimize the system as an effective policy in practice. Findings from this study may be useful for policymakers as well as district and school administrators in other countries who are working to develop and improve evaluation processes and outcomes.

References

American Educational Research Association (AERA). (2015). AERA statement on use of value-added models (VAM) for the evaluation of educators and educator preparation programs. Educational Researcher, XX(X), 1-5. doi:10.3102/0013189X15618385 Retrieved from http://edr.sagepub.com/content/early/2015/11/10/0013189X15618385.full.pdf+html American Statistical Association (ASA). (2014). ASA statement on using value-added models for educational assessment. Alexandria, VA. Retrieved from https://www.amstat.org/policy/pdfs/ASA_VAM_Statement.pdf Amrein-Beardsley, A. (2014). Rethinking value-added models in education: Critical perspectives on tests and assessment-based accountability. New York, NY: Routledge. Braun, H. I. (2005). Using student progress to evaluate teachers: A primer on value-added models. Princeton, NJ: Educational Testing Service. Retrieved from http://www.ets.org/Media/Research/pdf/PICVAM.pdf Bruner, J. S. (1991). The narrative construction of reality. Critical Inquiry, 18, 1-21. Bruner, J. S. (2002). Making stories: Law, literature, life. New York, NY: Farrar, Straus and Giroux. Bruner, J. S. (2004). Life as narrative. Social Research, 71(3), 691-710. Collins, C., & Amrein-Beardsley, A. (2014). Putting growth and value-added models on the map: A national overview. Teachers College Record, 116, 1-32. Harris, D. N. (2011). Value-added measures in education: What every educator needs to know. Cambridge, MA: Harvard Education Press. Weisberg, D., Sexton, S., Mulhern, J., & Keeling, D. (2009). The Widget Effect: Our national failure to acknowledge and act of differences in teacher effectiveness (2nd ed.). Brooklyn, NY: The New Teacher Project (TNTP). Retrieved from http://tntp.org/ideas-and-innovations/view/the-widget-effect

Author Information

Noelle Paufler (presenting / submitting)

University of North Texas, United States of America

Search the ECER Programme

Search for keywords and phrases in "Text Search"
Restrict in which part of the abstracts to search in "Where to search"
Search for authors and in the respective field.
For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.