External Examinations As Referents For Teachers’ Summative Assessment Practices

Author(s):

Walther Tessaro(presenting / submitting)

Conference:

ECER 2014

Network:

09. Assessment, Evaluation, Testing and Measurement

Format:

Paper

Session Information

09 SES 08 C, Teachers’ Roles in Assessment Practices

Paper Session

Time:

2014-09-04

09:00-10:30

Room:

B009 Anfiteatro

Chair:

Caroline Long

Contribution

More and more countries adopt the Anglo-Saxon tradition of curriculum-based external examinations using standardized tests (Kellaghan and Greaney, 2001). If the major aims of these tests can be different (Blais, 2005; Bain, 1996), they always contribute, in one way or another, to regulate teachers’ assessment practices (Ntamakiliro & Tessaro, 2010).

Many North American and European studies have showed the effects of testing on teaching practices (eg, Mons, 2009). These effects are particularly significant when it comes to high stakes tests (Amrein & Berliner, 2002).

For school administration, external examinations are a good way to regulate teaching practices by forcing teachers -to some extent- to "complete the program" within time limits and to comply with the core curriculum (Bain, 1996). They also serve as a "didactic lever" about teaching practices (Bodin, Delory, Lafontaine and Wolf, 2001), especially when there is a change of curriculum. Finally, they inform teachers about the skill levels expected from their students.

Indeed, the level of learning objectives in curricula is often too general to guide teachers in assessing their students’ learning (Monseur & Demeuse, 2005; Scallon, 2004). Thus, translating objectives into tasks that reveal on the one hand the level of achievement and on the other hand the definition of evaluation criteria (Roegiers, 2010), can be arbitrary.

In agreement with Allal (2009) and Laveault (2009), and as indicated by teachers (Dierendonck, 2008; Soussi, Guilley, Guignard & Nidegger, 2009), we believe that teachers’ summative assessment practices -considered as a professional judgement (Allal, 2012)- and construction methods of standardized tests using theories of measurement (Blais, 2005), should not be hierarchical but seen as complementary.

Teacher’s professional judgment convenes various categories of information. This multireferentiality (Mottier Lopez & Allal, 2008) combines internal and external referents, some of which are common to all students and others balanced with regard to singular situations of students.

Like Slomp (2009), in a comprehensive and situated way, we try to understand the dimensions on which testing enforced by a school system can guide teachers’ assessment practices. We wish to study the forms of reasoning and interpretation as well as the modalities of action shown by teachers when they compare their judgments to standardized tests. We also intend to explore how they articulate them with their internal referents during a process of social moderation (Wyatt-Smith, Klenowski & Gunn, 2010), aimed at reducing differences between teachers and between schools and thus ensuring greater equity in summative assessment practices.

Our research therefore questions the alignment (Biggs, 1999) between assessment prescriptions as conveyed by testing and teachers’ summative assessments practices in the classroom.

Method

This “training research” (Mottier Lopez, Tessaro, Dechamboux & Morales Villabona, 2012) involves 28 primary school teachers of the canton of Geneva (Switzerland), working in different schools with students representative of the cultural and socio-economic diversity of the canton‘s population. Their teaching experience was at least of three years in the concerned degrees and they had to follow their students during two years. A variety of professional experiences characterizes them: type of training, years of experience, career path, among others. The plan of research was replicated twice: a first cohort of teachers of third -consequently fourth- grade classes (students’ ages 8-10), and a second of fifth –consequetly sixth- grade classes (student’ ages 10-12). The research consists in three stages: (1) observations of correction practices in French and Mathematics, associated with semi-structured interviews; (2) social moderation meetings aiming to compare summative assessment practices and to build a common understanding of standards; (3) replication of part 1. In a participative approach (Bourassa, Belair & Knight, 2007), researchers and teachers jointly decided on the domains to be assessed by all teachers as well as other details of the research. During individual meetings (stages1 and 3), we systematically collected the summative assessment tools (written copies, referents, corrective controls) developed by each teacher. We also conducted semi-structured interviews to collect information on: teaching programs, assessed learning objectives, choice of items and scales in particular. Each teacher then corrected four copies of students likely to encounter difficulties, while verbalizing emerging interpretations and reasonings, according to a "thinking aloud" methodology, in a qualitative analysis approach (e.g. Charters, 2003). A final semi-structured interview phase ended the meeting. All meetings were recorded and then transcribed. In stage 2 we proposed a social moderation process, setting an explicit goal of professional development associated with our scientific research. This was aimed at promoting confrontation and a social construction of consensus. All verbal interactions were recorded and partially transcribed. All written data (semi-collective, collective and individual) were collected for analysis and methodological triangulation.

Expected Outcomes

The first analyses of recordings show the importance, for teachers, of an alignment between curriculum and didactic activities in the classroom. But teachers find it difficult to use a core curriculum that insufficiently directs them in the field of assessment of their students’ learning. External examinations guide the construction of teachers’ own summative assessment practices on several aspects. For example, they prescribe the presence of complex items, aimed at developing student competences in relatively open activities (Allal, Bétrix Köhler, Rieben, Rouiller Barbey, Saada-Robert & Wegmuller, 2001). They also indicate the most important objectives for teachers to remember in planning their teaching and, therefore, in assessing it. Regulation of practices is partly achieved in a differentiated manner, according to domains. Thus, in writing, testing has promoted the use of reference manuals and guides. It has also, over the years, demonstrated the importance of a preliminary situation scenario. According to teachers, the high proportion of problem-solving items in mathematics has been an important vector in changing their practices. Integrated to the teachers’ correcting guides, specification tables contain important elements for the analysis of students' productions: in addition to learning objectives, they indicate the importance and rating of items and the minimum expected performance. According to the many examples of instruments used by teachers, this formalization largely contributes to homogeneous summative assessment practices in schools. These first results encourage us to continue our investigations. Other findings will be presented at the congress, illustrated by examples drawn from our corpus.

References

Allal, L. (2009). Pratiques évaluatives des enseignants face aux méthodologies des recherches évaluatives portant sur les systèmes scolaires. In L. Mottier Lopez & M. Crahay (Ed.), Évaluations en tension : entre la régulation des apprentissages et le pilotage des systèmes (pp. 29-45). Bruxelles : De Boeck. Allal, L. (2012). Teachers’ professional judgement in assessment: a cognitive act and a socially situated practice. Assessment in Education: Principles, Policy & Practice, DOI: 10.1080/0969594X.2012.736364 Amrein, A. L. & Berliner, D. C. (2002). High-stakes testing, uncertainity, and student learning. Education Policy Analysis Archives, 10(18). Biggs, J.B. (1999). What the student does: Teaching for quality learning at university. Buckingham: Open University Press. Bodin, A., Delory, C., Lafontaine, D. & Wolf, J.-L. (2001). Quelle évaluation externe des élèves pour quels socles de compétences ? Pédagogies, 14, 56-65. Bourassa, M., Bélair, L. & Chevalier, J. (2007). Les outils de la recherche participative. Education et Francophonie, XXXV (2), 1-11. Charters, E. (2003). The use of think-aloud methods in qualitative research. An introduction to think-aloud methods. Brock Education, 12, 68-82. Kellaghan, T. & Greaney, V. (2001). Using assessment to improve the quality of education. Paris : UNESCO. Mons, N. (2009). Les effets théoriques et réels de l’évaluation standardisée. Réseau Eurydice. Monseur, C. & Demeuse, M. (2005). Les évaluations externes permettent-elles une régulation efficace ? In M. Demeuse et C. Monseur (Ed.), Vers une école juste et efficace. Bruxelles: De Boeck. Mottier Lopez, L. & Allal, L. (2008). Le jugement professionnel en évaluation : un acte cognitif et une pratique sociale située. Revue suisse des sciences de l’éducation, 3, 465-482. Mottier Lopez, L., Tessaro, W., Dechamboux, L. & Morales Villabona, F. (2012). La modération sociale: un dispositif soutenant l’émergence de savoirs négociés sur l’évaluation certificative des apprentissages des élèves. Questions vives, 18. Ntamakiliro, L. & Tessaro, W. (2010). Vers plus d’épreuves externes : une standardisation de l’évaluation scolaire? In P. Gilliéron Giroud & L. Ntamakiliro (Ed.), Réformer l’évaluation scolaire : mission impossible ? (pp. 165-185). Berne: Lang. Roegiers, X. (2010). L’école et l’évaluation. Bruxelles: De Boeck. Scallon, G. (2004). L’évaluation des apprentissages dans une approche par compétences. Bruxelles : De Boeck. Soussi, A., Guilley, E., Guignard, N. & Nidegger, C. (2009). Evaluation des acquis des élèves à l’école obligatoire. Genève : Service de la recherche en éducation. Slomp, D. (2009). Limiting Literacy, Limiting Excellence? An Examination into the Consequences of Large-Scale Standardized Literacy Assessment. Requête au Conseil de recherche en sciences humaines du Canada. Wyatt-Smith, C., Klenowski, V., & Gunn, S. (2010). The centrality of teachers’ judgement practice in assessment: a study of standards in moderation. Assessment in Education : Principles, Policy & Practice,17(1), 59-75.

Author Information

Walther Tessaro (presenting / submitting)

University of Geneva

Sciences of Education

Geneva

Search the ECER Programme

Search for keywords and phrases in "Text Search"
Restrict in which part of the abstracts to search in "Where to search"
Search for authors and in the respective field.
For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.