Comparing the Reading Frameworks of International and Hungarian Large-scale Studies

Author(s):

Péter Balkányi(presenting / submitting)

Conference:

ECER 2012

Network:

09. Assessment, Evaluation, Testing and Measurement

Format:

Paper

Session Information

09 SES 06 C, Relating National to International Large-scale Assessments

Parallel Paper Session

Time:

2012-09-19

15:30-17:00

Room:

FCT - Seminario 2

Chair:

Monica Rosén

Contribution

The goal of this paper is to compare three frameworks of reading literacy in order to support the planned modifications in the framework of Hungary's yearly national assessment, the National Assessments of Basic Competencies (NABC). The reading clusters of the NABC, PIRLS and the reading clusters of PISA are all instruments for assessing students' reading proficiency, however their frameworks do differ. The framework of PIRLS was the first of the three, with predecessors as early as the '70, and defined reading abilities in relation with both the purpose of reading and the processes of comprehension and the proportions of these in an actual test. PISA in the beginning of the new millennium had slightly different foundations given the different goal of the study. While PIRLS assesses the efficiency in learning to read of children in the fourth year of schooling, PISA examines the extent to what 15 year old students are probably able to succeed in the labour market with the aid of their reading skills acquired in mostly school.

The Hungarian NABC in Hungary came to existence in 2001 and gives account of the reading and mathematical performances of students in grades 6, 8 and 10 on a yearly basis since 2004. The aim of this study is to provide feedback to schools and maintainers about the average reading abilities of students compared to those of other schools, taking into account the ESCS background and previous results of students. Besides the study gives student-level feedback accessible to teachers and parents that can be analysed with the aid of an application provided online. The framework was formed accordingly. In order to permit the comparison of students' results, every student of a given population solves the same items.

Method

While PISA uses multiple dimensions to categorise texts like text format, reading situation, text type and medium, although only text format is considered in reporting, PIRLS and the NABC basically uses only one dimension, purpose for reading. There are also differences, even if only slight in defining task aspects. The paper addresses the question of the NABC's framework modification. What are the possibilities and what is the theoretical basis of these modifications? One proposition is to use more dimensions in the categorisation of texts, the other is already in function: the results of the three populations are reported on a common scale which makes the tracking of individual students' results possible.

Expected Outcomes

The paper gives an extensive account on the differences in the frameworks of three reading literacy assessments and about the differences in the ways these studies interpret and report about reading abilities. It becomes apparent that different goals demand different frameworks and these permit to explore possible ways to propose modifications to the framework of the NABC. The refinement of the framework would probably make the NABC a more comprehensive and unified assessment system and give way to multiple possibilities to compare students’ results. Besides the improvements in the assessment system is likely to facilitate the further advancement of Hungarian school system, shedding light on areas in the need of development.

References

Mullis, I.V.S., Martin, M.O., Kennedy, A.M., Trong, K.L., Sainsbury, M. (2011). PIRLS 2011 Assessment Framework. Chestnut Hill, MA: Boston College. PISA 2009 Technical Report. http://www.oecd.org/document/19/0,3746,en_2649_35845621_48577747_1_1_1_1,00.html PISA 2009 Assessment Framework - Key Competencies in Reading, Mathematics and Science. http://www.oecd.org/document/44/0,3343,en_2649_35845621_44455276_1_1_1_1,00.html Vári, P. (1997). MONITOR ’95. A tanulók tudásának felmérése [MONITOR ’95. Assessment of Knowledge of the Students.] Országos Közoktatási Intézet, Budapest. Balázsi, I., Felvégi, E., Rábainé Szabó, A., Szepesi, I. (2006). Országos Kompetenciamérés 2006. Tartalmi keret [National Assessment of Basic Competencies 2006. Assessment Framework.]. suliNova Kht., Budapest. Balkányi, P. (2009). Szövegértés tartalmi keret és feladatok bemutatása [Presentation of Framework and Items of Reading Assessment]. http://www.oh.gov.hu/letolt/okev/doc/orszmer/szovegertes_tartalmi_keret.pdf Országos Kompetenciamérés 2010. Feladatok és jellemzői [Items and their Characteristics Reports].(2011). Oktatási Hivatal Közoktatási Mérési Értékelési Osztály, Budapest.

Author Information

Péter Balkányi (presenting / submitting)

Educational Authority

Budapest

Search the ECER Programme

Search for keywords and phrases in "Text Search"
Restrict in which part of the abstracts to search in "Where to search"
Search for authors and in the respective field.
For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.