Indicators of School and Teacher Quality from TIMSS and PIRLS 2011

Author(s):

Ina Mullis(presenting / submitting)Michael Martin(presenting)

Conference:

ECER 2012

Network:

09. Assessment, Evaluation, Testing and Measurement

Format:

Paper

Session Information

09 SES 04 A, Findings from International Comparative Achievement Studies: Relating Achievement to Teaching, School and System Variables

Parallel Paper Session

Time:

2012-09-19

09:00-10:30

Room:

FCT - Aula 11

Chair:

Martin Goy

Contribution

For TIMSS and PIRLS 2011, considerable effort was devoted to developing background scales that were well-supported in the research literature, long enough to be scaled using IRT (at least 20 data points), and were related to effective teaching and learning. That is, could TIMSS and PITLS 2011 report data on robust context questionnaire scales about important school and teacher factors. The purpose of this presentation is to show how well several of these newly developed scales relate to achievement internationally.

The first newly developed scale to be presented, School Support for Academic Success, addresses school effectiveness research findings that academic emphasis is a key variable in explaining student achievement (McGuigan and Hoy, 2006). Research also has shown that students have higher achievement if they are safe at school (Konishi, et al.). Consistent with evidence that bullying is an increasing problem, another newly developed scale is Students Bullied at School (Carroll-Lind, 2009). Education studies have struggled to link student achievement to different types of instructional activities, but the concept of “Student Content Engagement” provided a promising avenue for TIMSS and PIRLS 2011. Student content engagement focuses on the importance of the aims of the activity that brings the students and the subject matter content together. This cognitive interaction may take the form of listening to a lecture, or rehearsing a set of facts equally as much as solving non-routine problems. The effectiveness of the student’s in-the-moment cognitive interaction with instructional content is what is important. Two new scales were developed to measure this construct from the perspective of teachers and of students, Teachers Make Efforts to Engage Students and Students Engaged in Mathematics. Having teachers that can provide leadership is another dimension of effective schools, and it may be that policies to reduce teacher attrition are needed in order to improve students’ achievement (Boyd, et al., 2009). Therefore, another newly developed scale measures Teacher Career Satisfaction.

Method

For each of the newly developed indicators, the presentation first will show all of the items that make up the IRT scale. To interpret the scale scores, the results also are presented by categories of the scale (high, medium, low) that are defined in terms of the responses to the scale items. For example, students assigned to the highest category have the scale score that would correspond to agreeing to all of the item statements in the scale. For each indicator, 2011 international averages of the percentages of students in each category (high, medium, and low) will be shown for PIRLS, TIMSS fourth grade, and TIMSS eighth grade. The percentages will be presented together with average achievement, internationally, for students in the category. For this presentation, each country’s achievement distribution will be set to have a mean of 500 and a standard deviation of 100. This will place the focus on achievement differences between categories of the background indicator, on average, across the countries that participated in TIMSS and PIRLS 2011. The number of countries was 52 in TIMSS fourth grade, 45 in TIMSS eighth grade, and 49 in PIRLS 2011, including most European countries.

Expected Outcomes

At both fourth and eighth grades, students had higher average achievement if they attended schools with a high level of School Support for Academic Success. Average achievement was lower for students bullied more often, particularly for those for whom it was a regular, monthly occurrence. Fortunately, most students reported that being Bullied at School was a rare event, occurring never or a few times a year for 70 percent of fourth grade and 81 percent of eighth grade students. Looking within classrooms, Teachers Make Efforts to Engage Students was positively related to student achievement. While most students had teachers that make efforts to engage students, the small percentage that did not have such teachers had lower achievement. About 80 percent of the fourth grade students reported being engaged in their lessons compared to only 60 percent of the eighth grade students. Teacher Career Satisfaction, also was positively related to achievement. About 80 percent of the students at both grades had teachers satisfied with their careers, but at the eighth grade, there was a substantial difference in achievement between students with very satisfied teachers and those whose teachers were only somewhat or not satisfied.

References

Boyd, D., Grossman, P., Lankford, H., Loeb, S., & Wyckoff, J. (2009). Who leaves? Teacher attrition and student achievement. CALDER Working Paper No. 23. Washington, DC: National Center for Analysis of Longitudinal Data in Education Research. Carroll-Lind, J. (2009). School safety: An inquiry into the safety of students at school. Wellington, New Zealand: Office of the Children’s Commissioner. Konishi, C., Hymel, S., Zumbo, B.D., & Li, Z. (2010). Do school bullying and student-teacher relations matter for academic achievement?: A multilevel analysis. Canadian Journal of School Psychology, 25, 19-39. McGuigan, L. & Hoy, W.K. (2006). Principal leadership: Creating a culture of academic optimism to improve achievement for all students. Leadership and Policy in Schools, 5, 203-229. McLaughlin, M., McGrath, D.J., Burian-Fitzgerald, M.A., Lanahan, L., Scotchmer, M., Enyeart, C., & Salganik, L. (2005). Student content engagement as a construct for the measurement of effective classroom instruction and teacher knowledge. Washington, DC: American Institutes for Research.

Author Information

Ina Mullis (presenting / submitting)

Boston College

TIMSS & PIRLS International Study Center

Chestnut Hill, MA

Michael Martin (presenting)

Boston College

TIMSS & PIRLS International Study Center

Chestnut Hill

Search the ECER Programme

Search for keywords and phrases in "Text Search"
Restrict in which part of the abstracts to search in "Where to search"
Search for authors and in the respective field.
For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.