Laboratory of Thinking – a Long Term Studies on Polish Students’ Skills

Author(s):

Marcin M. Chrzanowski(presenting / submitting)Krzysztof Spalik Piotr Walicki Barbara E. Ostrowska

Conference:

ECER 2014

Network:

09. Assessment, Evaluation, Testing and Measurement

Format:

Paper

Session Information

09 SES 13 C, Evaluating Science Education

Paper Session

Time:

2014-09-05

11:00-12:30

Room:

B009 Anfiteatro

Chair:

Marjeta Doupona

Contribution

1. Objectives

The aim of the study was to establish a reliable diagnosis of science knowledge level according to the Polish Science Curriculum in graduate students at ISCED 2. The need of such a diagnosis stems from the shortage of similar dependable large-scale studies in Poland.

2. The main research question

The study was focused on clarifying whether the introduction of the New Curriculum influenced students’ skills and if it did, in what aspects and to what extent.

3. Overview

Poland started its educational reforms in 1999. Since this year the previous dual structure of education (eight years of primary school and three or four years former secondary school cycle with three tracks) was replaced by a new tripartite one with six years of primary school, three years of lower secondary and three years of upper secondary school [1]. In 2008 a New Curriculum was introduced in Poland [2]. The Programme for International Student Assessment (PISA) study coincided with the educational reforms [3]. In year 2000 skills of 8-year primary school graduates were measured. In year 2003 PISA assessment was undertaken on second year graduates at lower secondary school (after the reform), and in 2012 it included students taught according to the New Curriculum. Thus the analysis of students’ performance during these years may give indirect information about the first effects of the reforms and the implementation of the New Curriculum in Poland. Polish students’ scores in science competencies rose form 498 points in 2006 through 508 in 2009 up to 526 in 2012 [4, 5, 6].

Which the New Curriculum emphasizes is improving students’ performance in the ability of using their knowledge together with critical and creative thinking skills, and not merely recalling information. In order to verify the actual outcomes of this reform and its practical implementation and execution, the Science Section, Educational Research Institute, launched in 2010 a study called Laboratory of Thinking – Diagnosis of Science Education in Poland. What is so special about this research is that it gives the possibility to analyze students’ performance in strict connection with the national curriculum in Poland. In Poland there are external exams (external assessment) but the items used for them are disclosed every year. Therefore, it is difficult to compare students’ results year by year using the same items . Laboratory of Thinking is meant to make up for this shortcoming.

Method

The study consists of four cycles and three of them have already been completed. In the first cycle in 2011 about 7,000 students who had been taught according to the former science core curriculum were assessed. In the second cycle in 2012 assessment was carried out on 7,000 students who had been taught according to the new science core curriculum for three years. In 2013 third cycle has been performed. The study covered 14,000 students at two different levels: those who had completed ISCED 2 level, and at the same time the students right after completing the first grade of ISCED 3 stage. In 2014 in the last fourth cycle about 14,000 students at two different levels will be assessed. Similarly to the students, the school for the study have been chosen through randomization process. The sample assessed is representative and proportional for the students’ population composition in ISCED 3 schools in Poland. In the study in question 208 standardized items were used. They consisted of tasks measuring complex skills, simple skills and knowledge in four science subjects: biology, chemistry, physics, geography, as well as reasoning in science with special attention paid to the methodology of experiments. All the items were prepared during workshops held for item authors since 2008 [7]. The average item preparation time was of about one year. Each item underwent several reviews as well as standardization and cognitive laboratory assessment if necessary. Together with the standardized testing tools student questionnaires concerning, among other things, the forms and methods used in sciences lessons by teachers working at ISCED 2 stage were used. After compiling results, the answers were subjected to statistical analysis. Evaluation of the answers was carried out by means of Classical Test Theory (CTT), as well as Item Response Theory (IRT). Item difficulty, Item discrimination as well as graphical analysis were performed. Using Rasch model and based on students’ skills levels six students’ proficiency levels were defined. The ranges were delimited so that the middle of the third range contained the mean (500), together with two standard errors of 100. In four standard errors 95% of results are contained and within those four standard errors six groups equal in number were defined.

Expected Outcomes

The study was conducted to answer the question regarding the influence of the New Curriculum introduction on Polish students’ skills. The study makes a supposition that the students’ knowledge level is related to the interpretation of the New Curriculum by their teacher, selection of forms and methods for teaching the aims outlined in the Curriculum, and development of students’ scientific understanding [8]. As far as the two first cycles are concerned, in the second one a statistically significant increase in students’ competences in measured skills for each of the science subjects was measured. It should be stressed that in the second cycle the study was conducted on students taught according to the New Curriculum. The New Science Core Curriculum was changed in the way that it the stress should be shifted from recalling information onto developing reasoning skills, and in the New Curriculum for biology some elements of experiment methodology were introduced. All science subjects were provided with a list of compulsory experiments to be performed. Both girls’ and boys’ scores increased in all three types of Polish secondary schools: liceum (comprehensive senior high school) technikum (a kind of senior vocational high school) and zasadnicza szkoła zawodowa (junior vocational high school). In 2011 as well as in 2012 little differences between girls and boys scores were noticed. These differences were not statistically significant. The results of the third study cycle of the study are being analyzed and prepared for comparative analyses with the results of the first two cycles. At a further stage of the study, in 2014, an analysis will be performed of shifts in the above-mentioned percentages at particular proficiency levels.

References

[1] Sara Bin Mahfooz, Kate Hovde, Alberto Rodriguez, Successful Education Reform: Lessons from Poland, key messages, ECA Knowledge Brief, 34, 2010 [2] Act of the Polish Parliament. Regulation of the Minister of Education. DzU 2008 Nr 4, poz. 17. – New Science Curriculum [3] PISA, Wyniki badania 2012 w Polsce, Programme For International Student Assessment, Organisation For Economic Co-Operation And Development http://www.ibe.edu.pl/images/badania/PISA_2012/SMALL_PISA_2012.pdf [4] PISATM 2006, Science Competencies for Tomorrow’s World, Volume 1 – Analysis, Organisation For Economic Co-Operation And Development, 2007 http://www.nbbmuseum.be/doc/seminar2010/nl/bibliografie/opleiding/analysis.pdf [5] PISA 2009 Results: Executive Summary, Organisation For Economic Co-Operation And Development, 2010 http://www.oecd.org/pisa/pisaproducts/46619703.pdf [6] PISA 2012 Results in Focus, What 15-year-olds know and what they can do with what they know, Organisation For Economic Co-Operation And Development, 2013 http://www.oecd.org/pisa/keyfindings/pisa-2012-results-overview.pdf [7] Barbara Ostrowska, Krzysztof Spalik (eds.), Umiejętności złożone w nauczaniu historii i przedmiotów przyrodniczych, Instytut Filozofii i Socjologii Polskiej Akademii Nauk, Warszawa 2010 [8] Laboratorium myślenia, Diagnoza umiejętności gimnazjalistów w zakresie przedmiotów przyrodniczych, Instytut Badań Edukacyjnych, Warszawa, 2012, http://beta.eduentuzjasci.pl/images/stories/publikacje/ibe-raport-laboratorium-myslenia-2011.pdf

Author Information

Marcin M. Chrzanowski (presenting / submitting)

Educational Research Institute

Science Section

Warszawa

Krzysztof Spalik

Educational Research Institute, Poland

Piotr Walicki