Reflections on a South African Model of Reading Effectiveness: Is the Underlying Framework a Square Peg in a Round Hole?

Author(s):

Surette van Staden(submitting)Sarah Jane Howie(presenting)Sarah Jane Howie (presenting)

Conference:

ECER 2012

Network:

09. Assessment, Evaluation, Testing and Measurement

Format:

Paper

Session Information

09 SES 03 A, Findings from International Comparative Achievement Studies: Relationships in Reading Performance (I)

Parallel Paper Session

Time:

2012-09-18

17:15-18:45

Room:

FCT - Aula 11

Chair:

Ina Mullis

Contribution

This paper reports on a South African study (van Staden, 2010) to identify and explain relationships between some major school- and student-level factors associated with successful reading at Grade 5. Hierarchical Linear Modeling (HLM) was used to determine the effect of a number of explanatory variables at student- and school level on reading achievement as outcome variable, while controlling for language using the South African Progress in International Reading Literacy Study (PIRLS) 2006 data. Utilizing Creemers’ Comprehensive Model of Educational Effectiveness (Creemers and Reezigt, 1999) as theoretical point of departure, this paper will focus on differences in reading achievement scores that were largely left unexplained using Creemers’ model as conceptual framework for the study.

Creemers’ Comprehensive Model of Educational Effectiveness (Creemers & Kyriakides, 2006; Kyriakides, Campbell & Gagatsis, 2000) for schools was used as a point of departure for this study, as this model most had relevance to already existing reading achievement literature (Leino, Linnakyla and Malin, 2004; Fuchs and Woessmann, 2004; Chapman and Tunmer, 2003; D’Angiulli, Siegel, & Maggi, 2004).

This paper seeks to answer the following research questions:

1. In the absence of identifiable reading effectiveness models, can a school effectiveness model (in the form of Creemers’ Comprehensive Model of Educational Effectiveness) sufficiently be adapted to that of a reading effectiveness model?

2. To what extent does the presupposed model capture the PIRLS 2006 South African data adequately?

3. Should differences in reading achievement not be explained using Creemers’ as conceptual framework for this study, how should the model change to suit the South African landscape more adequately?

The objective of this paper is therefore to establish the extent to which a model like Creemers’ Comprehensive Model of Educational Effectiveness could provide explanations for reading performance in a developing context and its ability to capture the PIRLS 2006 data adequately.

Bos (2000), in his TIMSS investigation into the benefits and limitations of large-scale international comparative achievement studies, adopted Creemers’ model for the purposes of the study. He employed the same four structural levels suggested by Creemers, but revised the components of quality, time and opportunity to suit the needs of his investigation.

A similar approach was followed for the purposes of this study, where Creemers’ Model of Educational Effectiveness was revised to constitute a model of reading effectiveness based on data provided by the Progress in International Reading Literacy Study (PIRLS) 2006.

For the purposes of the current study, a decision was also made to make adaptations to the original Comprehensive Model of Educational Effectiveness as proposed by Creemers, rather than the newly revised Dynamic Model. The Dynamic Model makes provision for investigation across time with multiple times for data collection, but for the purposes of this study, the available cross-sectional data was collected at one particular time with no follow-up or repeat measures. The Comprehensive Model of Educational Effectiveness is well established and has been critically reviewed for its validity in studies of educational effectiveness.

Method

Methodology For purposes of this study, Hierarchical Linear Modeling (HLM) was used from observations and measurements obtained from the PIRLS 2006 project. Factors emanating from contextual questionnaires of Grade 5 learners, their home environment, their schools and classrooms were identified in conjunction with learners’ test scores on the PIRLS 2006 achievement tests. The aim of these analyses was to establish the relationships between one or more explanatory variables, in this case obtained from items in the contextual questionnaires at learner and school level, and the outcome variables, i.e. reading achievement scores for the different language groups. The analysis of the PIRLS 2006 achievement and questionnaire data followed a confirmatory approach, therefore instead of using all variables from the different questionnaires, only a selection of variables that were expected to be related to reading literacy achievement was used for analysis purposes. In this way, the study was not guided by the available data alone, but rather by existing research into what is known about the factors that are likely to influence learner achievement. In doing so, a theoretical framework guided the analysis of data. Therefore, for the purposes of this study, a confirmatory rather than an exploratory method was chosen.

Expected Outcomes

Results In this study, Creemers’ Comprehensive Model of Educational Effectiveness was used as theoretical point of departure, as it supported preliminary ideas about reading achievement most closely. Most of the differences in reading achievement scores across five language groupings could not be explained using the conceptual framework for this study. The framework did not fit patterns found for the African language groupings (Nguni, Sotho and Tshivenda), and may therefore suggest changes to the framework to suit the South African landscape more adequately. The framework was insufficient in explaining the performance of children using African languages as Language of Learning and Teaching (LOLT) and little evidence could be found for factors where statistically significant effects were expected. This lack of evidence suggests that the current contextual data for African learners is insufficient to give evidence about associations and that stronger association and more data is required for the model to be tested effectively for the African language groupings. The conceptual framework was more appropriate in explaining reading achievement scores for the Afrikaans and English language groupings. Significant effects of school socio-economic status were found for the Afrikaans and English groupings, corroborating literature that infers the influence of socio-economic status on student achievement.

References

Bos, K. Tj. (2002). Benefits and limitations of large-scale international comparative achievement studies: The case of IEA’s TIMSS Study. Den Haag: CIP-Gegevens Koninklijke Bibliotheek. Chapman, J.W. and Tunmer, W.E. (2003). Reading difficulties, reading related self-perceptions and strategies for overcoming negative self-beliefs. Reading and Writing Quarterly, 19, 5-24. Creemers, B.P.M. (2008). Chapter 4: Empirical testing of the Comprehensive Model of Educational Effectiveness: Review of Research. In B.P.M Creemers & L. Kyriakides (eds.), The dynamics of educational effectiveness: A contribution to policy, practice and theory in contemporary schools (pp.48-72). London, UK: Routledge. Creemers, B.P.M and Reezigt, G.J. (1999). The concept of vision in educational effectiveness theory and research. Learning Environments Research, 2, 107-135. Creemers, B.P.M. & Kyriakides, L. (2006). Critical analysis of the current approaches to modelling educational effectiveness: The importance of establishing a dynamic model. School Effectiveness and School Improvement, 17 (3), 347-366. D’Angiulli, A., Siegel, L.S. & Maggi, S. (2004). Literacy instruction, SES and word-reading achievement in English-language learners and children with English as a first language: A longitudinal study. Learning Disabilities Research and Practice, 19(4), 202-213. Fuchs, T. and Woessmann, L. (2004). What accounts for international differences in student achievement? A re-examination using PISA data. CESifo Working Paper No. 1235. Kyriakides, L., Campbell, R.J. & Gagatsis, A. (2000). The significance of the classroom effect in primary schools: An application of Creemers’ Comprehensive Model of Educational Effectiveness. School Effectiveness and School Improvement, 11(4), 501-529. Leino, K., Linnakyla, P. & Malin, A. (2004). Finnish students’ multiliteracy profiles. Scandinavian Journal of Educational Research, 48(3), 251-270. Van Staden, S. (2010). Reading between the lines: Contributing factors that affect Grade 5 learner reading performance. (Unpublished doctoral dissertation). University of Pretoria, Pretoria, South Africa.

Author Information

Surette van Staden (submitting)

University of Pretoria

Science, Maths and Technology Education

Pretoria

Sarah Jane Howie (presenting)

University of Pretoria

Centre for Evaluation and Assessment

Pretoria

Sarah Jane Howie

University of Pretoria

Centre for Evaluation and Assessment

Pretoria

Search the ECER Programme

Search for keywords and phrases in "Text Search"
Restrict in which part of the abstracts to search in "Where to search"
Search for authors and in the respective field.
For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.

Session Information

Contribution

Method

Expected Outcomes

References

Author Information

Search the ECER Programme

Navigation

Info for