In 1994-95 the Third International Mathematics and Science Study (TIMSS) was conducted, under the auspices of the International Association for the Evaluation of Educational Achievement (IEA), at five grade levels in more than 40 countries (the third, fourth, seventh, and eighth grades, and the final year of secondary school). Students were tested in mathematics and science and information about the teaching and learning of mathematics and science was collected from students, teachers, and school principals. In 1998-99, the study was repeated (and is known as TIMSS-Repeat) in the upper grade level of the two grade levels containing most of the 13-year-old students, which for most of the 38 countries was grade 8. TIMSS-R provides countries that participated in TIMSS in 1994-95 the opportunity to monitor trends in mathematics and science education at the eighth grade between 1995 and 1999 from an international perspective. Continuing the approach of previous IEA studies, three conceptual levels of curriculum are distinguished in TIMSS and TIMSS-R: the intended, implemented and attained curriculum. At each level data was collected. The eighth- grade students completed an achievement test in mathematics and science (attained curriculum) and a questionnaire about their classroom experiences, attitudes towards mathematics and science, and home background. The achievement test was developed on the basis of a rotated test design (meaning that items were distributed across eight test booklets and were not administered to all the students) to increase the number of test items included in the test, and contained items in both the multiple choice and the open-ended format. In scoring of students' responses to open-ended items two- digit codes were applied with rubrics specific to each item or task. The first digit designated the correctness level of the response. The second digit, combined with the first, represented a diagnostic code used to identify specific types of approaches, strategies, or common errors or misconceptions. The mathematics and science teachers of the tested classes completed questionnaires about their academic preparation and instructional practices (implemented curriculum). School principals provided information about school characteristics and resources. At the level of intended curriculum a curriculum questionnaire was administered to curriculum experts and a test curriculum matching analysis (TCMA) was carried out. In the TCMA the appropriateness of each test item was judged by curriculum experts in every country. In addition, other system-level information was provided by each participating country. Most of the papers in this year's symposium will focus questions related to background variables and/or how these variables are influencing the learning of mathematics and science. Abstracts of the papers to be presented in the symposium follow below. Sufficient time for discussion will be allocated between paper presentations and at the end of the symposium. Given the number of papers, I propose two slots for this symposium. PAPER 1 The effects of schooling conditions on pupils' performance in science in South Africa Sarah Howie, Elsie Venter, Vanessa Scherman, Centre for Evaluation and Assessment, University of Pretoria, South Africa South Africa participated in TIMSS in 1995 and again in 1998. However, no data on school or teacher level could be analysed to provide the context for the students' poor achievements in science in 1995. From the TIMSS 1999 study, data is available at both school and teacher level in addition to the student level data permitting researchers to explore the reasons for the poor South African performance in science. In this paper, questions on school level regarding the leadership of the school, the physical conditions within the schools, the students' behaviour, the schools' expectations of parents as well as information regarding teachers will be investigated in relation to science achievement. As more than 80% of the South African TIMSS 1999 sample comprised schools, which are disadvantaged in terms of human and physical resources, it is critical that an analysis of the data from the principal's questionnaire is done to ascertain the effect of the conditions within these disadvantaged schools on the students' science performance. Partial Least Square analysis is applied to analyse both the indirect and direct effects of school-level variables on achievement PAPER 2 How to influence science attitudes in Australia, Canada, Cyprus and Korea Constantinos Papanastasiou & Elena C. Papanastasiou, The University of Cyprus The purpose of this study is twofold: 1.To examine the influence of parental and school factors on the science attitudes of eighth graders using the TIMSS-1999 data and the LISREL structural equation model. In particular, this study was designed to examine attitude predictors, focusing on those related to school and family. This study tried to answer the following questions. a. How do school climate and level of aspiration influence science teaching? b. What effect do school climate, educational background, teaching and aspiration have on student attitudes toward science? 2. To use the Cyprus model (derived from the TIMSS data) on the TIMSS data from Australia, Canada and Korea, to check for model fitness and to determined the strength of various predictors of attitudes toward science. According to the models for all countries, the strongest direct influence on attitudes towards science is that of teaching. The next strongest influence towards science attitudes was the level of aspiration exerted by the students, their families and their peers. Parental involvement and peers have direct and indirect effects on the students' science attitudes mediated through science teaching-activities. In general, the weakest effect was exerted by the school climate and educational background of the family. This indicates that in the countries examined in this analysis, the nature of the school climate, as well as the educational background of the parents did not significantly influence the science attitudes of the students. This is not necessarily a negative result though. These two factors, and especially that of educational background are not variables that could easily be manipulated in an attempt to change the attitudes of the students towards science. Consequently, low educational background of the family, and a bad school climate cannot necessarily hurt the attitudes of the students in science either. PAPER 3 Scoring trend items in TIMSS Juliane Barth, Ralph Carstens, The IEA Data Processing Center, Hamburg Large-scale assessments like TIMSS make use of constructed-response questions. In order to be used, responses given by students need to be scored, i.e. open answers provided by participants have to be transformed into a numerical equivalent. Since TIMSS is designed as a trend study, the same constructed-response questions are administered in more than one cycle of the study. The scorer reliability within a cycle and country is controlled by double scoring a subset of items. However, reliability within country across cycles was not controlled so far. In order to eliminate this deficiency, a software (Trend Scoring Reliability Software: TSRS) was developed which provided countries participating in TIMSS 2003 with electronic images of original answers given by students in TIMSS 1999. The software allows re-scoring the TIMSS 1999 constructed-response items by scorers used in TIMSS 2003. Scores obtained by this procedure can be used for comparison between the two cycles of the study. This will provide evidence whether or not scorers are more harsh or lenient in the second cycle. Making use if DIF-analyses, differential item behavior can be examined as well. In the presentation, the following topics will be addressed. " Design and implementation of TSRS " Analyses of difference on scale level " Analyses of differences on item level (DIF) If no data are available by the date of the conference, simulations will be used to show the possible outcomes. PAPER 4 Feedback to schools: communicating results of TIMSS-R multilevel analyses. Van den Broeck, A. & Van Damme, J., Catholic University of Leuven In 1999 38 countries participated in TIMSS-R (Third International Mathematics and Science Study-Repeat). Students solved mathematics and science items and students, teachers, and principals completed questionnaires. In Flanders, the TIMSS-R sample design differed from the general design: instead of taking one class in each of the selected schools, two classes per school were selected. Because of this design, we could implement a multilevel model with an intermediate level. An other national option in Flanders was the extension of the student's, teacher's, and principal's questionnaires with additional questions and an intelligence test for the students. We've also asked the parents of the students to complete a questionnaire. The Flemish dataset contains valid data of more than 4000 students in about 260 classes and about 130 schools. To explain the variance on the distinguished levels, multilevel analyses were carried out. Inspired by an article of Yang et al. (1999), individual school reports that presented some of the results of the multilevel analyses were compiled and sent to schools. Each report consists of three parts: ranking of the class based on the unadjusted mean score and the value added scores; graphs that represent the differential effectiveness of the class; and individual scores on the relevant dependent variables in comparison with mean scores. The first part of the report gives the position of the class compared to the other classes: first, the ranking is given on the basis of the achievement scores per se (both for mathematics and science). Secondly, the classes are ranked after adjusting for intake characteristics of the students. In all the graphs of this part, the uncertainty intervals are given to make it possible to compare with the overall mean. So, it can be determined whether a class is significantly more or less effective than can be expected given the intake characteristics of the students. In the second part of the reports, the relation between the achievement scores and the intelligence scores is represented by a regression line. Both the general regression line and the estimated regression line for the class (with a 95% confidence interval) are given in the graph. Moreover, each student of the class is represented. So, the teacher can check whether there is a student with an atypical achievement score given his or her intelligence score. The differential effectiveness graphs show whether the education is more or less effective for more or less intelligent students. The last part of the report contains the individual scores and the mean score on some variables, namely the variables that could reduce some of the variance in the multilevel analyses. The individual school reports are a mean to give the schools feedback based on the results of an international study. It's an attempt to communicate some meaningful information about the class to teachers and to principals (Yang et al., 1999) on the basis of the results of complicated multilevel analyses in school effectiveness research. Reference Yang M., Goldstein H., Rath T. & Hill N. (1999). The Use of Assessment Data for School Improvement Puposes. Oxford Review of Education, 25 (4), 469-483. PAPER 5 Looking for Cultural and Geographical Factors in Patterns of Responses to TIMSS Items Liv Sissel Grønmo, Marit Kjærnsli and Svein Lie, Department of Teacher Education and School Development, University of Oslo The main goals of international assessment projects like TIMSS are to establish reliable and valid scores for achievement that can be compared between groups and related to various background and context variables. In addition, the detailed item-by-item results provide an opportunity for a closer investigation of the similarities and differences between groups of countries. Our procedure will start by grouping countries according to similarities in response patterns across items. Countries in one group tend to have relative strengths and weaknesses in the same items. This grouping will be done separately in mathematics and science, as well as for the two subjects together. Focus will be on groups that make up meaningful groups in a geographical, cultural or political context. Preliminary analysis shows that interpretable groups of countries can be established: Nordic, English-speaking, East-European, East-Asian and (weakly established) South European groups of countries. For each of the country groups (e.g. Nordic countries, countries) it will be analysed what characterise particularly "favoured" and "unfavoured" items (i.e. items with particularly high or low percentage correct responses). In addition, these patterns will be compared and analysed in a context of curricular and cultural similarities and differences. International data from population 2, upper grade, for both math and science will be used in the analyses. Methods will include factor and discriminant analysis. PAPER 6 Consequences of overgeneralizing international comparisons. The case of Cyprus. Elena C. Papanastasiou, University of Kansas; Michalinos Zembylas, Michigan State University The data obtained from high school seniors for the Third International Mathematics and Science Study (TIMSS) for the country of Cyprus might appear to be contradicting. Although the Cypriot students did not perform well in mathematics in elementary, middle school, and in the non- advanced sectors of high school, the students in the advanced mathematics courses managed to perform exceptionally well. This was used as a basis to criticize TIMSS, by suggesting that the Cypriot students might have cheated on this test in population 3. The purpose of this study was to examine whether it was possible for an educational system that did not allow students to perform well relatively to other countries in mathematics, to produce students who could perform exceptionally well when they were high school seniors. The statistical analyses used as the basis for this paper were those of logistic regression. The results show that the Cypriot students who take advanced mathematics courses are significantly different from the students who do not do so in terms of background, attitudinal and instructional characteristics. We thus find nothing surprising or contradictory in the Cyprus results, for two reasons: (a) because of tracking and how it operates in Cypriot schools and (b) because of the nature of the two rather different TIMSS tests in the subject area of mathematics.