Participation in out-of-school activities is positively associated to better school attendance, more positive attitudes towards school work, better work habits, better interpersonal skills, lower drop-out rates, less time spent in unhealthy behaviours, and improved grades (Clark, 1988; Hamilton & Klein, 1998; McLaughlin, 2000; Posner & Vandell, 1994, 1999). Children learn most of their STEM knowledge or skills outside of school because they spent a lot of time outside of the classroom, leaving time for informal science learning opportunities (Falk & Dierking, 2010). Informal educational settings such museums and science centers, libraries, zoos and aquariums, afterschool, summer, and extended learning settings such as hobby clubs, community organizations, universities, parks, gardens, nature centers, and arboretums and public science events and festivals have been providing STEM learning opportunities to children for many years.

At the same time, the debates about what determines achievement in STEM school subjects in primary schools in EU still persist. The results of Croatian students on PISA and TIMSS assessments show low or average results in science and mathematics compared to results in other countries in the EU (Braš Roth et al, 2008; NCVVO, 2012a; 2012b). Also, the achievement of Croatian students in science and mathematics is low in comparison to their achievement in other school subjects (Burušić, Babarović & Šakić, 2008).

More recently, researchers are pointing out the importance of learning opportunities during the out-of-school hours.  Because STEM learning and teaching in out-of-school time are still relatively new, research and data collection relating specifically to STEM learning in out-of-school time and school achievement are limited, and even less data is available about the relationship of participating in STEM out-of-school activities and school achievement in children with different family background. It is a general expectation that students participating in out-of-school activities will achieve better success at school than students who do not participate in such activities. Previous research does not clearly confirm this expectation. Because of different research methods, it is difficult to judge the effectiveness of out-of-school time on school achievement.  Experimental studies conducted by James-Burdumy et al. (2005) and Gibson & Chase (2002) did not find a consistent positive relationship between participating in out-of-school activities and school achievement. Other studies using different methods have provided a clear positive correlation between student participation in STEM out-of-school activities and school achievement in the STEM area (Froschi et al., 2003; Hansen, et. al., 1995; Lee, 1997). In one of the few meta-analyses including 56 studies of out-of-school time programs, researchers found that out-of-school activities can have positive effects on STEM achievement and that out-of-school time strategies need not focus solely on academic activities to have positive effects on student achievement (Lauer et al., 2006).

According to the research presented above and the possible effects of out-of-school activities on students’ achievement, the aim of this study is to consider students STEM achievement on detailed manner and relate achievement primary to engagement in out-of-school STEM activities.

Participants The participants in this study were 1798 primary school students attending grades 4 to 6 (N4th grade586, N5th grade=580, N6th grade=632) and are equally distributed by and gender (48.6% girls. This study is a part of a broader research project JOBSTEM - STEM career aspirations during primary schooling. The study is conducted in 16 schools in the city of Zagreb and its surroundings. Within each school, two classes of students within one generation were sampled by random and joined the survey. Data used in this study were collected within regular classes in school. Measures STEM out-of-school activity was measured with STEM out-of-school activity scale containing 12 most common activities within the STEM area in which children can be involved during leisure time. The questions are adapted from the ROSE project (Relevance of Science Education) (Jenkins & Pell, 2006) taking into account the context and possible activities that children can participate. Respondents estimate on a 5-point Likert scale (1- almost never to 5- very often) how often they participate in each. Using confirmatory factor analysis, a one-factor structure of the scale was confirmed. Adequate validity was obtained with alpha coefficients between .83 (in fourth grade) and .86 (in fifth grade). STEM achievement was measured with knowledge tests designed for this study and with school grades. Three tests were designed to measure knowledge in STEM area, for grades 4, 5 and 6, following the outlines for STEM education and task design and current curriculum resources. In every test unidimensional structure was obtained using confirmatory and exploratory factor analyses. Acceptable reliability was obtained for all three tests (α4th = .78; α5th =.70; α6th = .79) (Dević, Babarović, Glasnović, Burušić, in press). Student school grades in school subjects related to STEM (Mathematics, Nature, Technical education, Geography, Biology) were collected directly from the school administration. Within a particular school grade, the average STEM achievement was calculated as the arithmetic means of the grades in STEM-related school subjects which were transformed into z-values and centered around the corresponding grade mean. Z-scores for 5th and 6th graders were then combined into a single scale measuring students’ achievement in STEM. Family SES was operationalized as the composite of parents’ education (2 questions), the student's assessment of how their family lives with current income compared to peers (1 question) and material possession in the household (9 questions) and was used to categorize students in the group of high or low SES.

The relationship between school out-of-school activities and school achievement is not very high (r=.15; p<.05) and it is moderated by student’s traits. Gender moderated the link between out-of-school activity and school achievement (Fisher Z=1.67; p<.05). Participating in out-of-school activities is more related to getting better school grades in STEM area in the group of males (r=.20; p<.01) than it is in the group of females (r=.12; p<.01). This moderation effect of gender on the relationship between participating in out-of-school activities and school achievement is found only when achievement was measured with school grades but not when achievement was measured as a STEM knowledge test score. Moderation effect of SES was not found regardless of the operationalization of student school achievement (school grades or STEM knowledge score). These findings may, in part explain the variability of findings regarding the correlation between participating in STEM out-of-school activities and STEM achievement found in the literature. It seems that the variability in findings depends not only on the theoretical framework or the used constructs but also on how those constructs are defined. Keith (2002) explains that one is likely to find slightly larger effects for influences on learning when learning is measured with grades in school as opposed to achievement scores because grades are more amenable to short-term change and in contrast, achievement test scores are more stable and more resistant to the short time change and intervention. The present findings suggest that whether participation in out-of-school activities is associated with school grades does not so much depend on SES, but it depends upon the gender of the student. Possible explanations and further implications are discussed within a European context.

Braš Roth, M., Gregurović, M., Markočić Dekanić, A. & Markuš, M. (2008). PISA 2006. Prirodoslovne kompetencije za život. Zagreb: NCVVO - PISA centar. Burušić, J., Babarović, T., & Šakić, M. (2008). Vanjsko vrednovanje obrazovnih postignuća u osnovnim školama Republike Hrvatske: Učenici 8. razreda, školska godina2007/2008., istraživački izvještaj. Zagreb: NCVVO/ Institut društvenih znanosti Ivo Pilar. Clark, R.M. (1988). Critical factors in why disadvantaged children succeed or fail in school. New York, NY: Academy for Educational Development. Dević, I., Babarović, T., Glasnović, D., Burušić, J. (in press). Development and Validation of New Objective Knowledge Tests in STEM Field for Primary School Students. Croatian Journal of Education. Falk, J. H., & Dierking, L. D. (2010). School is not where most Americans learn most of their science. American Scientist, 98(6), 486. Froschl, M., Sprung, B., Archer, E., & Fancsali, C. (2003). Science, Gender, and Afterschool: A Research-Action Agenda. Academy for Educational Development. Gibson, H.L. and Chase, C. (2002) Longitudinal Impact of an Inquiry-Based Science Program on Middle School Students' Attitudes toward Science. Science Education, 86, 693-705. Hamilton, L.S. & Klein, S.P. (1998). Achievement test score gains among participants in the foundations school-age enrichment program. Santa Monica, CA: RAND Corp Hansen, S., Walker, J., & Flom, B. (1995). Growing smart: What’s working for girls in school. New York: AAUWEF. James-Burdumy, S., Dynarski, M., Moore, M., Deke, J., Mansfield, W., & Pistorino, C. (2005). When schools stay open late: The national evaluation of the 21st Century Community Learning Centers Program. Washington, DC: U.S. Jenkins, E. W., & Pell, G. (2006). The Relevance of Science Education Project (ROSE) in England: a summary of findings. Leeds: Centre for Studies in Science and Mathematics Education Lauer, P. A., Akiba, M., Wilkerson, S. B., Apthorp, H. S., Snow, D., & Martin-Glenn, M. L. (2006). Out-of-school-time programs: A meta-analysis of effects for at-risk students. Review of educational research, 76(2), 275-313. Lee,V. (1997). Gender equity and the organization of schools. In B. Bank & P. Hall (Eds.) Gender, equity and schooling. New York: Garland Publishing, Inc. McLaughlin, M. (2000). Community counts: How youth organizations matter for youth development. Washington, DC: Public Education Network. Posner, J.K. & Vandell, D.L. (1994). Low-income children's after school care: Are there beneficial effects of after school programs? Child Development, 65, 440-456 Posner, J.K., & Vandell, D.L. (1999). After-school activities and the development of low-income urban children: A longitudinal study. Developmental Psychology, 25, 868-879.