Towards A Criterion To Judge The Correctness Of Transition Decisions
Author(s):
Ineke Pit-ten Cate (presenting / submitting) Thomas Hörstermann
Conference:
ECER 2012
Format:
Poster

Session Information

09 SES 05.5 PS, General Poster Exhibition

General Poster Session during Lunch

Time:
2012-09-19
12:30-14:00
Room:
FCEE - Poster Exhibition Area
Chair:

Contribution

Although pupils are confronted with many transitions during their school years, the transition from primary to secondary school, may be most significant. In several European countries entry into secondary schools is based on a selection process whereby teachers together with other professionals and parents decide which track would be most suitable for the pupil. Irrespective of the rights and wrongs of a tracking system (see Gamoran, 1992; Jürges & Schneider, 2011; and Werfhorst & Mijs, 2010, for reviews), the extent to which teachers are able to correctly identify pupils for orientation towards different tracks is not only important for the direct school choice, but may have a long lasting effect as the orientation decisions have a major impact on future adult life (Brunner & Martin, 2011). Although several studies have compared teacher based judgments of achievement with (standardized) test scores (e.g.  Demaray & Elliott 1998; Hoge & Coladarci, 1992), whereby generally moderate to high correlations are reported, relatively little is known about the validity of transition decisions.  In the Netherlands the transition decision is made based on a test score (“Cito-eindtoets”) combined with advice of the teacher and parental preference. Based on figures of the test in 2005, Stroucken, Takkenberg and Béguin (2008) reported that 83% of pupils were orientated towards the track indicated by their test score, whilst 13% started on a higher track and 5% on a lower track. After the first year in secondary school, three percent of pupils who followed the advice were orientated towards a higher track, and 3% towards a lower track. Of the pupils who chose a higher track than indicated, 8% was orientated to a lower track, whilst of the pupils choosing a lower than indicated track, 13% were orientated towards a higher track (Stroucken, Takkenberg & Béguin, 2008). In Germany and Luxembourg however, the orientation towards different tracks is more dependent on teacher judgments and although guidelines indicate these should be based on academic achievement and general skills (motivation, schoolattitude), research has shown other variables to affect judgments, whereby pupils with an immigration background and/or lower socioeconomic status are underrepresented in the higher tracks (Bos, 2004; Maier 2007; OECD, 2010). Hence, it can be concluded that when decisions are based on teacher judgments, these are error prone but could be improved by increasing diagnostic competence of teachers.  Our project aims to develop and compare different intervention methods to improve the diagnostic competence of primary school teachers in Luxembourg. However, in order to evaluate the effectiveness of our intervention modules, it is necessary to define a criterion for ‘right’ versus ‘wrong’ decisions. The first step of our intervention program therefore was to define such criterion and test its predictive validity in regards to transition decisions.  

 

Method

Data for 2696 Luxembourgish pupils in the last year of primary school (demographic characteristic, school marks, test results, and orientation advice) were used to select the most salient predictors of school-type (track). Results of this regression analysis showed that achievement grades and test results for German, French and Maths were all significantly associated with the school advice. After randomly splitting the data file in half, we z-transformed scores for each of these variables for the first half and computed the relative deviation of the combined academic achievement score for each track. Based on the relative deviations we were able to identify which track most fitted the pupils’ academic profile, i.e. the smaller the deviation, the more typical the pupils’ profile for that track). We then repeated this procedure for the second half of the dataset using the means and standard deviations retrieved for pupils in the 1st half. With this procedure we were not only able to define typical and a-typical cases but could also compare our typology with the actual orientation advice. Chi-square analyses revealed that the criterion held for both halves of the dataset, with a similar distribution of typical and atypical cases, correctly and incorrectly classified.

Expected Outcomes

Defining a criterion to judge the correctness of (transition) decisions is critical when evaluating intervention programs to increase the diagnostic competence of teachers. By applying statistical techniques to compute an integrated academic achievement score which can be translated into a pupil’s profile, will provide information regarding the typicality of that pupil in relation to a certain track. Our results show that such criterion can be successfully used to identify typical and atypical cases. The level of typicality then serves as a measure of correctness of the orientation decision and hence may facilitate the evaluation of training programs. One advantage of such criterion is that it is based on the combined attributes of pupils, identified as salient predictors of future academic success. Another advantage may be that the criterion is relatively free of bias, i.e. no demographic characteristics are incorporated (granting one cannot exclude potential bias in teachers’ marking). Although demographic variables may be taken into account when making final decisions on school choice, judgments regarding the academic potential of pupils may be best indicated by actual performance. Further analyses are needed to establish prognostic validity of the criterion.

References

Bos, W., Voss, A., Lankes, E.M., Schwippert, K., Thiel, O., & Valtin, R. (2004). Schullaufbahnemphelungen von Lehrkraften fur Kinder am Ende der vierten Jahrgangsstufe. In W. Bos, E.-M. Lankes, M. Prenzel, K. Schwippert, R. Valtin & G. Walther (Hrsg.), IGLU. Einige Länder der Bundesrepublik Deutschland im nationalen und internationalen Vergleich (S. 191-228). Münster: Waxmann. Brunner, M, & Romain, M. (2011). Die MAGRIP-Studie (1968-2009). Wie beeinflussen sozio-kognitive Merkmale von Kindern im Grundschulalter und ihre Bildungswege ihr späteres Leben als erwachsenen in Luxembourg? Luxembourg: Universität Luxembourg, Forschungseinheit EMACS. Demaray, M.K., & Elliott, S.N. (1998). Teachers’ judgments of students’ academic functioning: A comparison of actual and predicted performances. School Psychology Quarterly, 13, 8-24. Gamoran, A. (1992). The variable effects of high school tracking. American Sociological Review, 57, 812-828. Hoge, R.D., & Coladarci, T. (1992). Teacher-based judgments of academic achievement: A review of the literature. Review of Educational Research, 59, 297-313. Jürges, H., & Schneider, K. ( 2011). Why young boys stumble: Early tracking age and gender bias in the German school system. German Economic review, 12(4), 371-394. Maiern, U. (2007). Systematic effects of teachers in transfer rates to different school types - an analysis using educational statistics [Systematische Lehrereffekte bei Übergangsquoten auf weiterführende Schulen: Eine Analyse bildungsstatistischer Daten]. Zeitschrift für Erziehungswissenschaft, 10, 271-284 OECD (2010). PISA 2009 Results: Overcoming Social Background – Equity in Learning opportunities and Outcomes (Vol II). http://dx.doi.org/10.1787/97892640951504-en Stroucken, L., Takkenberg, D., & Béguin, A. (2008). Cito and the transition from primary to secondary education [Citotoets en de overgang van basisionderwijs naar voortgezet onderwijs]. Sociaaleconomische trends,2e kwartaal 2008, CBS . Retrieved from http://www.cbs.nl/NR/rdonlyres/92FA624A-0870-4930-8DDF-868F3DEB65DC/0/2008k2v4p07art.pdf. Werfhorst, H.G. van de & Mijs, J.J.B. (2010). Achievement inequality and the Institutional Structure of Educational systems. Annual Review of Sociology, 36, 407-428.

Author Information

Ineke Pit-ten Cate (presenting / submitting)
University of Luxembourg, Luxembourg
University of Luxembourg, Luxembourg

Update Modus of this Database

The current conference programme can be browsed in the conference management system (conftool) and, closer to the conference, in the conference app.
This database will be updated with the conference data after ECER. 

Search the ECER Programme

  • Search for keywords and phrases in "Text Search"
  • Restrict in which part of the abstracts to search in "Where to search"
  • Search for authors and in the respective field.
  • For planning your conference attendance, please use the conference app, which will be issued some weeks before the conference and the conference agenda provided in conftool.
  • If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.