Educational effectiveness researchers continually demonstrate that characteristics of schools, classes and teachers are to a certain extent related to several student outcomes (e.g., Scheerens & Bosker, 1997; Teddlie & Reynolds, 2000). For instance academic emphasis in schools, meaning that academic achievement is stressed, is positively associated with student achievement in elementary, middle and high schools (Hoy, Tarter, & Hoy, 2006). In order to obtain relevant information about these environmental characteristics, student or teacher questionnaires are most frequently used (e.g., Freiberg, 1999). By means of these questionnaires students and teachers provide their perceptions about the class and/or school environment (e.g., Fraser, 1986; Fraser & Walberg, 1991; Halpin & Croft, 1962; Hoy, Tarter, & Kottkamp, 1991). Different approaches are available to construct class, teacher and school characteristics from the individual data of the questionnaires. Most researchers aggregate the scales composed through an exploratory factor analysis (EFA) of the individual data (Rowan, Raudenbush, & Kang, 1991). However, this method implies some drawbacks. First of all, this procedure ignores the dependency present in the data due to the hierarchical data structure (i.e., students nested in classes nested in schools), causing the correlations and covariances, and thus the interpretations, to be biased (Muthén, 1991). Secondly, one may also question whether the scales constructed at the individual level are also relevant at the group level (see Hox, 1993). Given these difficulties, a number of authors proposed the use of multilevel factor analysis which explicitly models the different levels present in the data (e.g., Longford & Muthén, 1992; Muthén, 1989, 1991, 1994; Rabe-Hesketh, Skrondal, & Pickles, 2004). The approach developed by Muthén (1989, 1990) is implementable in conventional software for covariance structure analysis, like Mplus (Muthén & Muthén, 1998-2006) and LISREL (Jöreskog & Sörbom, 1993). Although already a number of authors applied multilevel factor analysis (e.g., Duncan, Strycker, Duncan, & Okut, 2002; Holfve-Sabel & Gustafsson, 2005; Kaplan & Elliott, 1997; Toland & De Ayala, 2005; Yang, 2003), among which educational effectiveness researchers, this technique is far from consistently used when confronted with a nested data structure. Moreover, until now few studies showed a systematic comparison between the more traditional methods to construct characteristics at group level and multilevel factor analysis. Therefore, we propose to contrast the more traditional methods of an EFA and aggregation with the more recent approach of multilevel factor analysis, as ways to construct class, teacher and school characteristics. Specific attention is given to the following aspects: number of factors, interpretation and meaning of the factors, reliability and external validity. To this end, different datasets from two longitudinal research projects are used: the SiBO-project in primary education (Maes, Ghesquière, Onghena, & Van Damme, 2002) and the LOSO-project in secondary education (Van Damme, De Fraine, Van Landeghem, Opdenakker, & Onghena, 2002). Characteristics at class, teacher and school level are constructed (1) using EFA in SPSS (SPSS, 2005) with aggregation and (2) by means of multilevel confirmatory factor analysis (MCFA) using STREAMS 3.0 (Gustafsson & Stahl, 2005) and Mplus 4.0 (Muthén & Muthén, 1998-2006). Secondly, the external validity of these characteristics with regard to several school outcomes in students is investigated using multilevel regression analysis in MLwiN (Goldstein et al., 1998). Analyses of the data in secondary education will be presented. We hypothesize that MCFA will show significant improvement compared to the more conventional methods of EFA and aggregation. The proposed research project will have relevance to both educational effectiveness research in particular as well as to environmental research in general. References Duncan, S.C., Strycker, L.A., Duncan, T.E., & Okut, H. (2002). A multilevel contextual model of family conflict and deviance. Journal of Psychopathology and Behavioral Assessment, 24, 169-175. Fraser, B.J. (1986). Classroom environment. London: Croom Helm. Fraser, B.J., & Walberg, H.J. (1991). Educational environments: Evaluation, antecedents and consequences. Oxford: Pergamon. Freiberg, H.J. (1999). School climate: measuring, improving, and sustaining healthy learning environments. London: Falmer. Goldstein, H., Rasbash, J., Plewis, I., Draper, D., Browne, W., Yang, M., Woodhous, G., & Healy, M. (1998). A user's guide to MLwiN. London: University of London, Institute of Education, Multilevel Models Project. Gustafsson, J.-E., & Stahl, P.A. (2005). STREAMS 3.0 User's Guide. Mölndal, Sweden: MultivariateWare. Halpin, A.W., & Croft, D.B. (1962). The organizational climate of schools. Washington D.C.: US Department of Health, Education & Welfare: National Institute of Education. Holfve-Sabel, M.-A., & Gustafsson, J.-E. (2005). Attitudes towards school, teacher, and classmates at classroom and individual levels: An application of two-level confirmatory factor analysis. Scandinavian Journal of Educational Research, 49(2), 187-202. Hox, J. (1993). Factor analysis of multilevel data: Gauging the Muthén model. In J.H.L. Oud & R.A.W. van Blokland-Vogelesang (Eds.), Advances in longitudinal and multivariate analysis in the behavioral sciences (chapter 10: pp. 141-155). Nijmegen: ITS. Hoy, W.K., Tarter, C.J., & Kottkamp, R.B. (1991). Open schools/healthy schools: measuring organizational climate. Newbury Park (California): Sage. Hoy, W.K., Tarter, C.J., & Hoy, A.W. (2006). Academic optimism of schools: a force for student achievement. American Educational Research Journal, 43, 425-446. Jöreskog, K.G., & Sörbom, D. (1993). LISREL 8: User's reference guide. Chicago: Scientific Software International, Inc. Kaplan, D., & Elliott, P.R. (1997). A model-based approach to validating education indicators using multilevel structural equation modeling. Journal of Educational and Behavioral Statistics, 22, 323-347. Longford, N.T., & Muthén, B.O. (1992). Factor analysis for clustered observations. Psychometrika, 57, 581-597. Continuation of references Maes, F., Ghesquière, P., Onghena, P., & Van Damme, J. (2002). Longitudinaal onderzoek in het basisonderwijs. Van doelstellingen tot onderzoeksopzet [Longitudinal research in primary education. From objectives to research design]. LOA-rapport nr. 1. Leuven: Steunpunt LOA. Muthén, B.O. (1989). Latent variable modeling in heterogeneous populations. Psychometrika, 54, 557-585. Muthén, B.O. (1990). Mean and covariance structure analysis of hierarchical data. Los Angeles: University of California. Muthén, B.O. (1991). Multilevel factor analysis of class and student achievement components. Journal of Educational Measurement, 28(4), 338-354. Muthén, B.O. (1994). Multilevel covariance structure analysis. Sociological Methods & Research, 22(3), 376-398. Muthén, L.K. and Muthén, B.O. (1998-2006). Mplus User's Guide. Fourth Edition. Los Angeles, CA: Muthén & Muthén. Rabe-Hesketh, S., Skrondal, A., & Pickles, A. (2004). Generalized multilevel structural equation modeling. Psychometrika, 69, 167-190. Rowan, B., Raudenbush, S.W., & Kang, S.J. (1991). School climate in secondary schools. In S.W. Raudenbush & J.D. Willms (Eds.), Schools, classrooms and pupils: international studies of schooling from a multilevel perspective (chapter 14: pp. 203-223). New York: Academic Press. Scheerens, J., & Bosker, R.J. (1997). The foundations of educational effectiveness. Oxford: Elsevier Science. SPSS (2005). SPSS Base 14.0 User's Guide. Chicago: SPSS Inc. Teddlie, C., & Reynolds, D. (2000). The international handbook of school effectiveness research. London: Falmer Press. Toland, M.D., & De Ayala, R.J. (2005). Validity studies: A multilevel factor analysis of students' evaluations of teaching. Educational and Psychological Measurement, 65, 272-296. Van Damme, J., De Fraine, B., Van Landeghem, G., Opdenakker, M.-C., & Onghena, P. (2002). A new study on educational effectiveness in secondary education in Flanders: an introduction. School Effectiveness and School Improvement, 13(4), 383-397. Yang, Y. (2003). Measuring socioeconomic status and its effects at individual and collective levels: A cross-country comparison. Göteborg, Sweden: Acta Universitas Gothoburgensis.