Teacher effects among low, middle, and high ICT resource schools - multiple group, multilevel structural equation modeling
Author(s):
Moonsoo Lee (presenting / submitting) Soojin Kim Jung-A Han
Conference:
ECER 2016
Format:
Paper

Session Information

09 SES 03 B, Findings from ICILS 2013: Relating Individual, Class and School Characteristics to ICT Use and CIL Achievement

Paper Session

Time:
2016-08-23
17:15-18:45
Room:
NM-F103a
Chair:
Wilfried Bos

Contribution

Theoretical Framework

Many of the countries in the world are making efforts to improve their students’ achievement as well as their future competency by measuring the variety of competencies at the international level such as participating international studies of student assessment, for example PISA’s problem solving competency, IEA ICCS’s global citizenships and IEA ICILS’s Computer and Information Literacy.

ICILS 2013 is the first cross-national study to investigate the students’ acquisition of Computer and Information Literacy (CIL) as an essential skill in 21st century. The ICILS 2013 was implemented to measure CIL when students need to live in digital age and to gather information about the use of digital device and the related ICT capabilities, attitudes, as well as safety and security use. To measure CIL and the students several CIL related aspects and background information, the test modules and background questionnaires of students, teachers, principals, and ICT-coordinator were used at the study (Fraillon et al., 2013).

In order to clarify differences in educational attainment, several factors should be considered as an important impact at the different levels of education system. Most relevant factors can be the socio-economic status and self-efficacy of students, which is the basic information for students. In addition to these students’ variables, CIL score can be affected by the school contexts (e.g., school ICT resources, teachers’ ability using computer). Therefore, we need to investigate and understand the contexts related school level information, which effects on students’ CIL.

In the context of integrating new technologies in schools, factors contributing on either the teacher level or the school level are known to contribute to the use of ICT (Lorenz et al., 2015). Of course, the IT infrastructure in schools is an important factor regarding the use of ICT for learning purposes. Here, it seems to be well-accepted that ICT as such does not support learning itself but needs to be properly integrated into relevant teaching and learning strategies (Lai, 2008; Law et al., 2008). According to Eickelmann (2011) and ISTE (2009), there seems to be a range of supporting and hindering factors on the school level related to school leadership, attitudes and competencies of teachers as well as the ICT infrastructure in schools. As previous research reveals, that students’ ICT competency was affected by the level of ICT competency among the teachers and their willingness to use ICT in their lessons (Aoki et al., 2013, Law et al., 2008). These papers found that the ICT infrastructure of schools on its own is not enough to enhance the ICT competency but the efforts of the teachers and administration were more important than any other factor.

The purpose of the present study is to examine the relationship between the contextual variables and CIL scores. This study broadly examines several teacher effects to students’ achievement based on school ICT resource levels. In general, subgroups or subpopulations of schools may exist that differ substantially from the general population in ways that impact on the relationship between school level factors and student outcomes (Palardy, 2008). Therefore, this study is approached by analyzing the possible relationships of the context indicators using multiple group, multilevel Structural Equation Modeling (SEM) with CIL in an international comparative context. Here, it addresses the following research questions:

(1)   Do the effects of student level variables on CIL achieving differ in low, middle, and high ICT resource schools?

(2)   Do the effects of school level variables (especially teacher variables) on CIL achieving differ in low, middle, and high ICT resource schools?

Method

Methods Data Sources Data from the ICILS 2013 were used for this study. Approximately 60,000 students and 35,000 teachers from 18 countries were participated in the ICILS 2013. In Korea, 2,888 8th graders and 2,189 teachers from 150 middle schools were participated. For the secondary analysis, data is used from ICILS 2013 with students’ achievement data, data from students’ background questionnaires as well as the teachers’ and principals’ questionnaires considered. The sample of 150 schools was divided into three groups – low, middle, and high ICT resource schools (50 schools for each group) – based on the ICT resource composite scores. The composite scores were calculated using three variables - “ratio of school size and number of computers available for students”, “ICT resources at school”, and “computer resources at school”. On the student level, students’ SES background, “using specific ICT program”, “using ICT in daily life”, “ICT self-efficacy on basic skill”, “ICT self-efficacy on applied skill”, and “interest and enjoyment in using ICT” are taken into account. On the school level, “use of ICT for teaching at school”, “emphasis on teaching ICT skills”, and “collaboration between teachers in using ICT” are focused. Total 10 from student and school level variables were used to analyze teacher effects on CIL. SEM model using this study depicts direct effects between the following variables and CIL: ICT self-efficacy on basic skill, ICT self-efficacy on applied skill, and interest and enjoyment in using ICT for student level, use of ICT for teaching at school, emphasis on teaching ICT skills, and collaboration between teachers in using ICT for school level. Note that a direct effect is a relationship between a predictor and an outcome that does not involve the intervention of a moderator or mediating variable. The model also shows direct effects between several sets of variables (e.g., use of specific ICT program and ICT self-efficacy basic skill, use of ICT in daily life and ICT self-efficacy applied skill, students’ SES background and use of specific ICT program). Analytical Procedure As for analytical method applying the structural and hierarchical characteristics of the data, we create a multiple group, multilevel structural equation model of student and school, utilizing the statistical program M-Plus (Muthén & Muthén, 2012), and analyze the effects of educational context variables on the achievement in each level. A robust Maximum Likelihood estimator (MLR) was used to take the complex data structure into account (Muthén, 2004).

Expected Outcomes

Preliminary Results Our SEMs generally consisted of two levels, a within-school level (addressing effects at the student level within schools) and a between-schools level (addressing effects between schools), to account for the nested structure of the data. These effects of the two levels were estimated simultaneously for three ICT resource level groups. Since analyses presented here aimed to develop a multilevel model and to estimate effects at different levels, SEM analyses used covariance matrices. In this study, we used overall fit statistics, and incremental fit indices to evaluate model fit. Assessment of fit indicated that the multilevel model fit the theoretical model well. The χ² was relatively large and significant (χ²=38952.427, P=0.00) Next, incremental fit indices (TLI and CFI) also used. A rule of thumb for the incremental indices is that values greater than roughly .90 may indicate reasonable good fit of the researcher’s model (Hu & Bentler, 1999). The obtained TLI and CFI were placed around 0.95, indicating a reasonable model fit. Next, the obtained RMSEA were 0.05 and we assumed that values between 0.05 and 0.08 indicated reasonable error of approximation followed by Browne and Cudeck (1993). At the within-school level, students’ CIL score was regressed on “Interest and enjoyment in using ICT”, “ICT self-efficacy basic skill” and “ICT self-efficacy applied skill”, while the latter two were allowed to correlate. In addition, “Use of specific ICT program” and “Use of ICT in daily life” were assumed to predict “ICT self-efficacy basic skill” and “ICT self-efficacy applied skill”. At the between-school level, CIL score was regressed on “use of ICT for teaching at school” and on “emphasis on teaching ICT skills”, as well as on “collaboration between teachers in using ICT”. The further results will be discussed in detail at the full paper.

References

References Aoki, H., Kim, J., & Lee, W. (2013). Propagation & level: Factors influencing in the ICT composite index at the school level. Computers & Education, 60(1), 310–324. doi:10.1016/j.compedu.2012.07.013 Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In: K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136-162). Beverly Hills, CA: Sage. Eickelmann, B. (2011). Supportive and hindering factors to a sustainable implementation of ICT in schools. Journal for Educational Research Online, 3(1), 75–103. Fraillon, J., Schulz, W. & Ainley, J. (2013). International Computer and Information Literacy Study: Assessment framework. Amsterdam: International Association for the Evaluation of Educational Achievement (IEA). Hu, L.T., & Bentler, P.M. (1999), "Cutoff Criteria for Fit Indexes in Covariance Structure Analysis: Conventional Criteria versus New Alternatives," Structural Equation Modeling, 6 (1), 1-55. ISTE (2009). Essential Conditions. Necessary conditions to effectively leverage technology for learning. Retrieved from http://www.iste.org/docs/pdfs/netsessentialconditions.pdf Lai, K.-W. (2008). IT and the learning process. In J. Voogt & G. Knezek (Eds.), International handbook of information technology in primary and secondary education (pp. 215–230). New York: Springer. Law, N., Pelgrum, W. J., & Plomp, T. (Eds.). (2008). Pedagogy and ICT use in schools around the world: Findings from the IEA SITES 2006 study. Hong Kong: CERC-Springer. Lorenz, R., Eickelmann, B. & Gerick, J. (2015). What Affects Students’ Computer and Information Literacy around the World? – An Analysis of School and Teacher Factors in High Performing Countries. In D. Slykhuis & G. Marks (Eds.), Proceedings of Society for Information Technology & Teacher Education International Conference 2015 (pp. 1212-1219). Chesapeake, VA: Association for the Advancement of Computing in Education (AACE). Muthén, B. O. (2004). Latent variable analysis. Growth mixture modeling and related techniques for longitudinal data. In D. Kaplan (ed.), The sage Handbook of quantitative methodology (S. 345-368). Thousand Oaks, CA: Sage. Muthén, L. K., & Muthén, B. O. (2012). Mplus user's guide. 7th. Los Angeles, CA: Muthén & Muthén. Palardy, G. J. (2008). Differential school effects among low, middle, and high social class composition schools: A multiple group, multilevel latent growth curve analysis. School Effectiveness and School Improvement, 19(1), 21-49.

Author Information

Moonsoo Lee (presenting / submitting)
Korea Institute for Curriculum and Evaluation (KICE), Korea, Republic of (South Korea)
Korea Institute for Curriculum and Evaluation (KICE), Korea, Republic of (South Korea)
Korea Institute for Curriculum and Evaluation (KICE), Korea, Republic of (South Korea)

Update Modus of this Database

The current conference programme can be browsed in the conference management system (conftool) and, closer to the conference, in the conference app.
This database will be updated with the conference data after ECER. 

Search the ECER Programme

  • Search for keywords and phrases in "Text Search"
  • Restrict in which part of the abstracts to search in "Where to search"
  • Search for authors and in the respective field.
  • For planning your conference attendance, please use the conference app, which will be issued some weeks before the conference and the conference agenda provided in conftool.
  • If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.