Mediating Effects of Grade Repetition in the Explanation of Mathematical Literacy Performance from PISA 2012 Perspective

Author(s):

Pou-seong Sit(presenting / submitting)Kwok-cheung Cheung(presenting)Wai-cheong Cheong Soi-kei Mak Man-kai Ieong

Conference:

ECER 2014

Network:

09. Assessment, Evaluation, Testing and Measurement

Format:

Paper

Session Information

09 SES 02 A, Findings from PISA: Performance In and Attitudes Towards Mathematics

Paper Session

Time:

2014-09-02

15:15-16:45

Room:

B010 Anfiteatro

Chair:

Anabela Serrão

Contribution

Motivation of the study

In earlier studies conducted before 1970s, Jackson (1975) concluded that research evidence was not sufficient to support grade repetition is more beneficial than grade promotion. Later review conducted by Homes and Matthews (1984) revealed that grade promotion is more beneficial than grade repetition in terms of academic achievement, attitudes and personal adjustment. In a more recent review of the literature, Brophy (2006) asserted that school-imposed grade repetition is particularly stressful to students, and is often associated with reduced self-esteem, impaired peer relationships, and alienation from school. Although around the world hundreds of studies have been undertaken on grade repetition in the past decades, there is still a serious lack of research documenting the educational experiences of students who are not promoted, some of the cases are voluntary whereas most others school-imposed.

In PISA2012, Macao tops other 64 economies in grade repetition. About 43 percent of Macao’s 15-year-olds have repeated in primary and/or junior secondary grades. Males and those coming from disadvantaged homes are especially the victims of grade repetition. While it is not possible to make use of the PISA data to find out why students have undergone grade repetition, it is feasible to conduct mediational analyses on the test and questionnaire responses to unveil the processes and mechanisms underlying the effects of grade repetition on mathematical literacy performance. The processes or mechanisms, if identified, will illuminate the educational experiences of the struggling students.

Conceptual and research framework

The mediation process depicts a conceptual model as well as an analytical framework guiding the conduct of enquiry of the present study. The relationships amongst grade repetition (IV), mediational processes or mechanisms (MVs) and student’s mathematical literacy performance (DV) are epitomized in a figure documented in Baron and Kenny (1986).

Statistically speaking, a mediator (MV) carries the influence of an independent variable (IV) onto a dependent variable (DV). Pedagogically, it reveals the underlying educational processes or psychological mechanisms through which IV exerts its effect on DV. Four criteria must be met simultaneously for a mediation to happen: (1) the effect of IV on DV is statistically significant; (2) the effect of IV on MV is statistically significant; (3) MV has an independent statistically significant effect on DV after accounting for the effect of IV; and (4) the total effect of IV on DV shrinks upon the addition of the MV to explain DV together with it.

The 5 blocks of quality education indicators for mathematics education in Macao schools are potential candidates of MVs in the present study. The five blocks of the 23 MVs are:

Learning Mathematics – Familiarity with mathematical concepts, Experience with pure mathematics tasks at school, Mathematics self-efficacy, Mathematics self-concept, Mathematics interest, Instrumental motivation for mathematics, Mathematics work ethics, Mathematics behaviors, Subjective norms in mathematics, Mathematics anxiety, Attributions to failure in mathematics
Mathematics Experiences – Mathematics teachers’ classroom management, Cognitive activation in mathematics lessons, Mathematics teacher’s support, Disciplinary climate
Problem Solving Experiences – Perseverance, Openness for problem solving
Availability and Use of ICT– ICT resources, ICT use at home for school-related tasks
Classroom and School Climate – Teacher-student relationships, Sense of belonging to school, Attitude towards school: Learning outcomes, Attitude towards school: Learning activities

Hypothesis of the study

This study examines the mediating effects of Macao’s grade repetition. Five blocks of quality indicator for mathematics education in Macao schools are analyzed. The following hypothesis is postulated for statistical significance testing:

Five blocks of quality indicators for mathematics education in Macao schools, after controlling for the effects of the ESCS and gender, have differential mediating effects explaining the relationships between grade repetition and mathematical literacy performance.

Method

Sample The data for the present study is drawn from the Macao sample in PISA 2012 (OECD, 2012). The number of 15-year-old student sampled is 5,397, studying in 45 secondary schools in Macao. Dependent Variable The dependent variable of the proposed mediation analysis is mathematical literacy performance, which for each student is measured as a set of five plausible values (MATHPV1-5). Independent Variables The independent variable is grade repetition during primary and secondary education: repeat=0 (not repeated); repeat=1 (has repeated one time or more). There are two other independent variables serving as covariates in the mediation analysis, namely gender and ESCS of the home of the student. Gender is a dichotomous variable: female=0; male=1, whereas ESCS is a continuous variable standardized across the 34 OECD countries, with mean =0 and standard deviation =1. All independent and mediating variables are documented in details in OECD (2012). Mediating Variables The variables used as mediating variables in the present study are the 23 mathematics education quality indicators identified for use in Macao schools (Cheung, Sit, Mak, & Ieong, 2013, p.34). The Pearson correlation between these indicators with mathematical literacy performance ranges from 0.15 to 0.51 in absolute magnitude. Analysis Strategies WesVar 5.1 was used to examine the mediation of grade repetition effect on mathematical literacy performance after accounting the effect of gender and ESCS (WESTAT, 2007). The use of WesVar in statistical data analysis, on the one hand, takes care of the complex two-stage stratified sample design and, on the other hand, handles appropriately the rotated test forms of the PISA 2012 (OECD, 2005, 2013). In the present study, grade repetition of student is the IV, and student’s mathematical literacy performance is the DV. Variables hypothesized and tested for statistical significance for mediation effects include the 23 mathematics education quality indicators identified in Macao’s PISA2012. There are two control variables, namely gender and ESCS (i.e. Economic, Social and Cultural Status) of the students. The following tables summarize the research results of the present study: Table 1: Blocks of variables used in the mediational analysis Table 2: Summary of analyses of the 23 mediating variables Table 3: Mediating variable MATHEFF - Estimated standardized and unstandardized regression coefficients Table 4: Summary of violation of criteria of mediation analysis for statistical significance testing of mediating effects Table 5: Significance testing of indirect mediation effects in the explanation of the grade repetition performance gap in mathematical literacy

Expected Outcomes

The following abstract summarizes the expected research outcomes of the present study: Amongst all the 65 participating economies in PISA 2012, Macao is high up in the league table of grade repetition. The data shows that 43 percent of Macao’s 15-year-olds repeated one or more years in their primary and/or junior secondary years of education. Although it is not feasible to use the PISA data to ascertain the causes of grade repetition, it is possible to base on the student’s responses to the questionnaire to uncover the underlying processes and mechanisms mediating the effects of grade repetition on mathematics literacy performance. After adjusting for the effects of gender, ESCS (Economic, Social and Cultural Status) of the sampled students and some pertinent demographic characteristics, around ten mediating variables pertaining to student’s learning mathematics, mathematics experiences, problem-solving experiences, and availability and use of ICT have been identified. The total effect of grade repetition on mathematical literacy performance shrinks from 88 to 61 score points, equivalent to 0.7 grade level of progression. The joint mediating effects identified in the full model are only partial, mainly concerning with mathematics self-efficacy, self-concept, and anxiety, openness in problem solving, use of ICT in school-related tasks at home, disciplinary climate in class, as well as familiarity with mathematical concepts and experience with pure mathematics tasks appropriate to the 15-year-olds at school. Implications of study to the various stakeholders such as students, principals and curriculum designers are discussed within Macao’s local schooling and the broader international contexts.

References

Baron, R. M., & Kenny, D. A. (1986). The moderator mediator variable distinction in social psychological-research - Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51(6), 1173-1182. Brophy J. (2006). Education policy series 6: Grade repetition. Paris & Brussels: IIEP & IAE. Cheung, K.C., Sit, P.S., Mak, S.K., & Ieong, M.K. (2013). Macao-PISA 2012 Report: Assessment of mathematical, scientific and reading literacy performance of 15-year-old students from an international comparison perspective. Macao: Educational Testing and Assessment Research Centre, University of Macau. Eide, E., & Showalter, M. (2001). The effect of grade retention on educational and labor market outcomes. Economics of Education Review, 20, 563-576. Eisenmom, T. (1997). Reducing repetition: Issues and strategies. Paris: IIEP-UNESCO. Jimerson, S. (2001). Meta-analysis of grade retention research: Implications for practice in the 21st century. School Psychology Review, 30, 420-437. Homes, C.T. & Matthews, K.M. (1984). The effects of non-promotion on elementary and junior high school pupils: A meta-analysis. Review of Educational Research, 54(2), 225-236. Jackson, G.B. (1975). The research evidence on the effects of grade retention. Review of Educational Research, 45(4), 613-635. OECD (2005). PISA 2003 data analysis manual: SPSS users. Paris: OECD. OECD (2013). PISA 2012 results: What students know and can do: Student performance in mathematics, reading and science. Paris: OECD Publishing. WESTAT (2007). WesVar® 4.3 User's guide. Rockville: WESTAT.