Factor Structure of PISA 2006 Correlates

Author(s):

Jaan MikkKarin Täht

Conference:

ECER 2010

Network:

9. Assessment, Evaluation, Testing and Measurement

Format:

Paper

Session Information

09 SES 07 B, International Large-Scale Assessments of Student Performance I

Paper Session

Time:

2010-08-26

15:30-17:00

Room:

P673, Porthania

Chair:

Josef Basl

Contribution

Sipe and Curlette (1997) have synthesised the results of 103 meta-analyses and found that the most important factors of educational achievement were motivation (effect size .82), ability (.61), quality of instruction (.39), classroom social group (.30) and leisure time use (.30). However, at the national level, there are only some studies of correlates of educational attainment. For example, Waldfogel and Zhai (2008) have studied the relationship between educational expenditures and TIMSS; Lee (2009) was interested in the connection between math self-efficacy and PISA 2003; Lynn and Mikk (2009) have found a correlation between national intelligence and PISA 2006.

The aim of our research was to find a factor structure for the educational and economic indices of countries that correlate with the PISA 2006 results. The hypothesis was that the structure is similar to the one found by Sipe and Curlette.

Method

The data for the analysis were taken from PISA 2006 test results and questionnaires, from The World Fact Book (2008) and the book by Lynn and Vanhanen (2006). A correlation analysis and confirmatory factor analysis (CFA) were used to find the factor structure of the researched indicators. CFA was carried out with Lisrel 8.80 (Jöerskog and Sörbom, 2006). The following cut-off values of goodness-of-fit values are used: for SRMR .08 or below and for CFI .95 or greater (Brown, 2006). RMSEA ≤ .08 suggests an adequate model fit (Browne and Cudec, 1993). The relationships in the confirmatory factor analysis are correlations and their values have been transformed into effect sizes according to the formula proposed by Furr (2008, 6). Altogether, 37 characteristics of student motivation, study time, parent education, human development and national intelligence were included in the correlation analysis.

Expected Outcomes

The characteristics that had statistically significant correlation coefficients with the average PISA results were factor analysed. The fit indices of the confirmatory factor analysis were good: RMSEA = .080, SRMR = .03, CFI = .99. The factor analysis revealed four factors in the set of PISA 2006 results studied and their correlates: motivation to learn, quantity of instruction, human development and PISA results. The correlations of the factors with the PISA results were as follows: motivation .64 (effect size 0.83), quantity of instruction .63 (0.82) and human development .78 (1.26). National intelligence had a correlation of 0.84 with the PISA results. We see that the effect of student motivation was the same size as in Sipe and Curlette's meta-synthesis (.83 and .82 correspondingly). Student ability had a lower effect in individual-level studies (.61) than it had in our country-level study (1.56). The social factors had a large effect on the PISA results (1.26) but, in the meta-synthesis, the effect of classroom social group was .30. In the PISA study, the quantity of instruction was as effective as motivation (.82) but, in the meta-synthesis, the effect of the quality of instruction was .39.

References

Brown T. A. (2006). Confirmatory Factor Analysis for Applied Research. The Guilford Press. New York, London. Browne M. W. and Cudek, R. (1993). Alternate ways of assessing model fit. In K. A. Bollen and J. S. Long (ed.). Testing structural equation models, (136-162). Newbury Park, CA: Sage. Furr, R. M. (2008). Summary of Effect Sizes and their Links to Inferential Statistics. http://psych.wfu.edu/furr/EffectSizeFormulas.pdf Jöerskog K. and Sörbom D. (2006). LISREL 8.80. Scientific Software International, Inc. Lynn, R., and Vanhanen, T. (2006). IQ and Global Inequality. Georgia: Washington Summit Publishers. Lynn, R., and Mikk, J. (2009). National IQs predict educational attainment in math, reading and science across 56 nations. Intelligence, 37, 305-310. Sipe, T. A., Curlette W. L. (1997). A Meta-Synthesis of Factors Related to Educational Achievement: A Methodological Approach to Summarizing and Synthesizing Meta-Analyses. International Journal of Educational Research, 25(7), 583-698. The World Factbook (2008). https://www.cia.gov/library/publications/the-world-factbook/index.html Waldfogel, J., Zhai, F. (2008). Effects of public preschool expenditures on the test scores of fourth graders: evidence from TIMSS. Educational Research and Evaluation, 14 (1), 9–28.

Author Information

Jaan Mikk

University of Tartu

Department of Education

Tartu

Karin Täht

University of Tartu, Estonia

Search the ECER Programme

Search for keywords and phrases in "Text Search"
Restrict in which part of the abstracts to search in "Where to search"
Search for authors and in the respective field.
For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.