Patterns in English Language Proficiency – some Differences and Similarities between Seven European Countries

Author(s):

Lisbeth Åberg-Bengtsson

Conference:

ECER 2008

Network:

9. Assessment, Evaluation, Testing and Measurement

Format:

Paper

Session Information

09 SES 09A, Assessment in Language Subjects.

Paper Session

Time:

2008-09-12

10:30-12:00

Room:

AK2 139

Chair:

Tobias C. Stubbe

Contribution

Large-scale comparative surveys of foreign language proficiency have been sparse over the years, in contrast to frequent international studies of, for example, first language, mathematics and science, e.g., IEA, TIMSS and PISA studies. However, in 2002-2003 on behalf of the ‘European network of policy makers for the evaluation of educational systems’, a survey of English at the end of compulsory school was conducted in eight European countries, generating an impressive body of data, in a first step explored only to a very limited extent (Bonnet, 2004). Based on these data, a research project, granted by the Swedish Research Council and aiming at analysing variability in language proficiency, as well as in attitudes and language habits, is being carried out. The suggested paper presents a part study of that project and aims at exploring and describing, in some detail, differences and similarities in English ‘proficiency-profiles’ between seven countries (viz., Denmark, Finland, France, the Netherlands, Norway, Spain, and Sweden). Although Germany participated in the assessment project as the eighth country, its test results were not reported; consequently, they are not included in the present study. Influential contemporary psychometric researchers (e.g., Carroll, 1993) have stressed the hierarchical nature of achievement tests. By using powerful multivariate techniques that have been suggested during the last decades (e.g., Gustafsson, 2001; Loehlin, 2004), the complex of observed variables in tests of the kind involved in the present study can be decomposed into latent variables and their dimensionality investigated (see also, e.g., Åberg-Bengtsson, 1999, 2004, 2005; Åberg-Bengtsson & Ottosson, 2006). Such an approach is particularly important when differences in performance between groups of test-takers are investigated.

Method

Test results from around 11,000 students have been used in the present analyses. On the whole, subjects involved constitute a reasonably representative sample from each of the seven countries, although it needs to be emphasized that the study was not conducted altogether identically in each country, and, consequently, the results cannot be claimed to be fully comparable. The test was made up from 75 items covering the following skill areas: linguistic knowledge (25 items), written comprehension (16 items), written expression (21 items), and oral comprehension (13 items). The students got written instructions for the test in their own languages and a CD, common for all countries, was provided for the oral comprehension part of the test. In the analyses, the single items were gathered into nine passages that were in turn related to a number of postulated factors. Drawing on previous knowledge on factor structures of foreign language performance among Swedish compulsory school students (Åberg-Bengtsson & Erickson, 2006) and knowledge of the domain, confirmatory factor analysis (CFA), carried out with structural equation modelling techniques (SEM) applied to several-group modelling (see, e.g., Gustafsson & Stahl, 2000). Initially, a hypothesized four-factor model, common for the seven countries, was set up and tested. Next, the model was tested with respect to the different countries. As the model was, to some extent, further elaborated on, with respect to modification indices, the procedure was to a certain extent also ‘model generating’ (see, Jöreskog & Sörbom, 1993, p. 115). With an additional aim to investigate possible gender differences, the posited factors were regressed on sex.

Expected Outcomes

Preliminary results indicate some similarities between the seven countries but also several interesting differences with respect both to the factor structure and to gender differences.

References

Åberg-Bengtsson, L. (1999). Dimensions of performance in the interpretation of diagrams, tables, and maps: Some gender differences in the Swedish Scholastic Aptitude Test. Journal of Research in Science Teaching 36, 565-582. Åberg-Bengtsson, L. (2004). Do small rural schools differ? A comparative two-level model of reading achievement among Swedish 9-year-olds. Scandinavian Journal of Educational Research, 48, 19-33. Åberg-Bengtsson, L. (2005). Separating the quantitative and analytic dimensions of the Swedish Scholastic Aptitude Test (SweSAT). Scandinavian Journal of Educational Research, 49, 359-383. Åberg-Bengtsson, L., & Erickson, G. (2006). Dimensions of national test performance in language and mathematics: A two-level approach. Educational Research and Evaluation, 12, 469-488. Åberg-Bengtsson, L., & Ottosson, T. (2006). What lies behind graphicacy? Relating students’ results on a test of graphically represented quantitative information to formal academic achievement. Journal of Research in Science Teaching, 43, 43-62. Bonnet, G. (Ed.). (2004). The assessment of pupils’ skills in English in eight European countries 2002. Retrieved February 18, 2008 from http://cisad.adc.education.fr/reva/france/pubsomm3.htm Carroll, J.B. (1993). Human cognitive abilities: A survey of factor-analytic studies. New York: Cambridge University Press. Gustafsson, J.-E. (2001). On the hierarchical structure of intelligence and personality. In J. M. Collis & S. J. Messick (Eds.), Intelligence and personality: Bridging the gap in theory and measurement (pp. 25 – 42). Hillsdale, NJ: Erlbaum. Gustafsson, J.-E., & Stahl, P. A. (2000). STREAMS user’s guide: Version 2.5 for Windows. Mölndal, Sweden: MultivariateWare. Jöreskog, K.-G., & Sörbom, D. (1993). LISREL 8: Structural equational modeling with SIMPLIS command language. Chicago, IL: Scientific Software International. Loehlin, J. C. (2004). Latent variable models: An introduction to factor, path, and structural analysis (4th ed.). Hillsdale, NJ: Erlbaum.

Author Information

Lisbeth Åberg-Bengtsson

Mid Sweden University

Department of Education

Härnösand

186

Search the ECER Programme

Search for keywords and phrases in "Text Search"
Restrict in which part of the abstracts to search in "Where to search"
Search for authors and in the respective field.
For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.