Differential school effects: findings from Portugal and Brazil

Author(s):

Maria Eugenia Ferrao(presenting / submitting)

Conference:

ECER 2014

Network:

09. Assessment, Evaluation, Testing and Measurement

Format:

Paper

Session Information

09 SES 12 A, Investigating Regional Differences with Large-Scale Data

Paper Session

Time:

2014-09-05

09:00-10:30

Room:

B010 Anfiteatro

Chair:

Heike Wendt

Contribution

Researchers and policy makers mutually inform each other (e.g. Weiss, 1977, 1979). The instrumental and conceptual (Hofstetter & Alkin, 2003) uses of educational evaluation specially need the knowledge transference from science. In this way, there have been many developments in school effectiveness research (SER) over the last 40 years, which allowed the identification and incorporation of factors of effectiveness into school improvement programs, in such a way that lead to better learning outcomes for students (e.g. Cresswell, 2004). For instance, there is a broad consensus in the literature about the indicator of educational value added as a measure of institutional performance (OECD, 2008). Nevertheless, the need to conduct SER in different countries and contexts has been recognized since in the US, the UK, the Netherlands and Australia it is where it has been mostly conducted (Peng, Thomas, Yang, & Li, 2006). Reviews on SER in developing countries (e.g. Scheerens, 2001a, 2001b; Yu, 2007) are based on studies that fail to satisfy SER methodological requirements, being of limited use either for scientific purposes or for policy and practice. Some recent articles (Ferrão & Couto, 2013a, 2013b; Ferrão, 2014) have discussed SER issues concerning two Portuguese speaking countries, Brazil and Portugal. The papers include aspects related to the scientific properties (e.g. Teddlie, Reynolds, & Sammons, 2000), i.e. the existence of school effects, the magnitude, the consistency and the stability, but they have not addressed the topic of differential school effects. Are schools differentially effective for certain groups of students within school? Are school effects generalizable within schools? Are schools differentially effective across subject areas? These are some of the research questions addressed in this paper.

Multilevel models are applied considering student as level one unit and school as level two unit. Specifically, statistical models are specified as a traditional value-added model (Ferrão & Goldstein, 2009) and as a growth model (Ferrão, 2011). The models include student’s performance in maths/reading at the end of the year as the response variable and students’ prior achievement, socioeconomic status, gender, race/ethnic group as controlling variables.

Method

Portugal The population consists of students enrolled in primary, elementary or lower secondary education in the region NUT III of Cova da Beira. The target population is 7,729 students (3,418 in primary, 1,709 in elementary and 2,602 in lower secondary education). For the purpose of this paper primary education is considered. The sampling method was based on stratification by county and type of school, followed by conglomerate sampling within each stratum (random selection of schools and, within each school, random selection of classes). The educational census of 2004/5 was used as sampling frame. Data were collected at the beginning and at the end of academic years 2005/6, 2006/7 and 2007/8. Two cohorts of students were considered, covering nine grades and involving 86 classes in 45 schools in the primary education, 127 classes in 14 schools in elementary and lower secondary education. The sample is representative of the target population by agrupamento, by county and by NUT III region. Further details about the sampling design are presented by Vicente (2007). Brazil The population consists of students enrolled in the daytime 2nd grade in urban public schools, with at least 20 students per class, and in urban private schools, with at least 10 students per class, in the municipalities of Belo Horizonte (federal state of Minas Gerais), Campinas (federal state of São Paulo), Campo Grande (federal state of Mato Grosso do Sul), Rio de Janeiro, and Salvador (federal state of Bahia). The educational census of 2003 was used as sampling frame. The total number of schools in the municipalities was 3,097, the number of those involved in the study was 310, and the starting number of students 20,700. For the purpose of this study the municipality of Campinas was considered, involving 4,400 starting students in 60 schools. The sampling method was based on stratification by municipality and type of school (public state dependent, public municipal dependent, private), followed by conglomerate sampling within each stratum (random selection of schools and, within each school, random selection of classes). Data were collected at the beginning (March 2005) and at the end (November 2005) of grade 2, and then in November 2006, November 2007 and November 2008. Further details about the sampling design and data collection are presented by Brooke & Bonamino (2011). Ferrão (2014) provide further details on instruments and data collection.

Expected Outcomes

This paper provides findings of research on differential school effects based on longitudinal data modelling and discusses implications for educational evaluation in Brazil and Portugal. The results show a great deal of regional disparities regarding educational outcomes that are related to pupils’ socioeconomic and prior achievement heterogeneity. This is an on-going study. In September we will present results regarding the differential effects by gender, age and ethnic/race.

References

Brooke, N., & Bonamino, A. (Eds.). (2011). GERES 2005: Razões e Resultados de uma Pesquisa Longitudinal sobre Eficácia Escolar. Rio de Janeiro: Walprint. Retrieved from http://dc314.4shared.com/doc/DbyLSnul/preview.html Ferrão, M. E. (2011). Adjusting for measurement error in longitudinal multilevel models. In ISI World Statistics Congress. Dublin. Ferrão, M. E. (2014). School effectiveness research findings in the Portuguese speaking countries: Brazil and Portugal. Educational Research for Policy and Practice, 13(1), 3–24. doi:10.1007/s10671-013-9151-7 Ferrão, M. E., & Couto, A. (2013a). Indicador de valor acrescentado, tópicos sobre consistência e estabilidade: uma aplicação ao Brasil. ENSAIO: Aval.Pol.Públ.Educ, 21(78), 131–164. Ferrão, M. E., & Couto, A. P. (2013b). The use of a school value-added model for educational improvement: a case study from the Portuguese primary education system. School Effectiveness and School Improvement, in press, 1–17. doi:10.1080/09243453.2013.785436 Ferrão, M. E., & Goldstein, H. (2009). Adjusting for measurement error in the value added model: evidence from Portugal. Quality & Quantity, 43(6), 951–963. doi:10.1007/s11135-008-9171-1 Hofstetter, C. H., & Alkin, M. C. (2003). Evaluation use revisited. In T. Kellaghan & D. L. Stufflebeam (Eds.), International Handbook of Educational Evaluation (pp. 197–222). Dordrecht: Kluwer Academic Publishers. Peng, J., Thomas, S., Yang, X., & Li, J. (2006). Developing school evaluation methods to improve the quality of schooling in China: a pilot “value added” study. Assessment in Education: Principles, Policy & Practice, 13(2), 135–154. doi:10.1080/09695940600843252 Scheerens, J. (2001a). Monitoring school effectiveness in developing countries. School Effectiveness and School Improvement, 12(4), 359–384. doi:10.1076/sesi.12.4.359.3447 Scheerens, J. (2001b). Introduction School Effectiveness in Developing Countries. School Effectiveness and School Improvement, 12(4), 353–358. doi:10.1076/sesi.12.4.353.3443 Teddlie, C., Reynolds, D., & Sammons, P. (2000). The Methodology and Scientific Properties of School Effectiveness Research. In C. Teddlie & D. Reynolds (Eds.), The International Handbook of School Effectiveness Research (pp. 55–133). London and New York: Routledge/Falmer. Vicente, P. (2007). Plano Amostral do Projecto 3EM – Eficácia Escolar no Ensino da Matemática [Sampling Design of 3EM Survey - School Effectiveness in Maths]. In M. E. Ferrão, C. Nunes, & C. Braumann (Eds.), Estatística: Ciência Interdisciplinar (pp. 849–858). Sociedade Portuguesa de Estatística. Weiss, C. H. (1977). Using social research in public policy making. Lexington, MA: Books, Lexington. Weiss, C. H. (1979). The Many Meanings of Research Utilization. Public Administration Review, 39(5), 426–431. doi:10.2307/3109916 Yu, G. (2007). Research evidence of school effectiveness in sub-saharan Africa. Retrieved from http://www.edqual.org/publications/workingpaper/edqualwp7.pdf

Author Information

Maria Eugenia Ferrao (presenting / submitting)

University of Beira Interior

Mathematics

Covilhã

Search the ECER Programme

Search for keywords and phrases in "Text Search"
Restrict in which part of the abstracts to search in "Where to search"
Search for authors and in the respective field.
For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.