Evidence-Based School Counseling: An Analysis of the Student Success Skills
Author(s):
Conference:
ECER 2015
Format:
Paper

Session Information

09 SES 13 A, Evaluations and School Improvement

Paper Session

Time:
2015-09-11
11:00-12:30
Room:
326. [Main]
Chair:
Jan Van Damme

Contribution

School counseling is quickly expanding internationally. The implementation of school counseling reflects various organizational structures, depending on the country, region, and emphasis on an systemic approach versus a clinical mental health approach. No matter the systemic organization of school counseling as a profession, implementation of evidence-based interventions provides a strong foundation for positively impacting student outcomes. However, in many countries, a lack of evidence-based interventions exists. This presentation will introduce one evidence-based school counseling intervention validated in the United States, the Student Success Skills (SSS) Curriculum, and provides one example of how to evaluate the impact of the SSS intervention.

This project analyzed the connection between the success of the Student Success Skills (SSS) curriculum, currently being implemented as a normal part of the educational day of students around the United States and the meta-cognition functioning of the students (as assessed by the Junior Meta-Cognition instrument). The SSS is an evidence-based school counseling intervention currently delivered to all students in each of the schools used in this study.  However, each of the schools implemented SSS with various levels of fidelity.  The success and impact of the SSS on student achievement is well documented, however, the relationship between the curriculum and meta-cognition of student is less clear.  This study, using the Junior Meta-Cognition Awareness Inventory (Jr. MAI; Sperling, Howard, Miller, & Murphy, 2002) and the Student Engagement in School Success Skills instrument (SESSS; 2014), assessed correlations between student success and scores on the two instruments. 

The intervention (the Student Success Skills Curriculum) has been in use in the region for over two years in various school districts as part of the normal educational experience and will continue to be used after the conclusion of this study.  In other regions of the United States, SSS has been used for a decade.  The evaluation instruments, the Jr. MAI and the SESSS, have been used in Colorado and Florida to assess the impact of the SSS intervention.  The total number of participants in the study was 4,862, with some enrollment fluctuations affecting the final number of completed instruments.  The participants were students between the ages of 11 and 19 years old, enrolled in Middle School, Junior High, or High School in five schools in the Midwest region of the United States.  The study used a double-camouflaged system to protect the identities of participants. 

This study both looks at:

(a) The impact of an existing intervention delivered in each of these schools normal educational setting – the Student Success Skills (SSS) curriculum – impacts academic achievement.  Demographic factors normally collected by each school district and reported to the public will be analyzed in the study as well.  These factors are non-identifiable to specific students. 

(b) The correlation between student scores on the Junior Meta-Cognition Awareness Inventory (Jr. MAI) and scores on the Student Engagement in School Success Skills Instrument (SESSS) will be analyzed and the relationship between the scores and student achievement will be explored. 

Outcomes from this study may help school counselors and other educational leaders learn strategies for monitoring student progress, strategies for identifying barriers to learning, and strategies to increase their understanding of factors affecting student behavior.  An analysis of the outcome data can inform future decisions regarding goals of the school counseling program and the school improvement plan.  How students’ meta-cognitions correlate to their attitudes and knowledge about learning, their self-regulation strategies, and their support of classmates can inform future intervention designs to positively impact student academic and personal/social development related to classroom learning and achievement.

Method

The Junior Meta-Cognition Instrument (JMCI) and the Student Engagement in Success Skills Survey (SESSS) were used to determine the meta-cognitions conducted by students using the Students Success Skills (SSS; http://www.studentsuccessskills.com) in several middle and high schools in the Midwest in the United States. Almost 5000 students participated in the study. The study has a pre-test and two-post-tests using the two instruments mentioned above. The pre-test was given at the beginning of the school year (August), while post-test one was given at the holiday break (December) and post-test two was given at the end of the school year in spring (April-May). The research questions included: a. Do correlations exist between scores on the JMCI and the SESSS in relation to student academic growth? b. Do correlations exist between academic growth and growth on the SESSS? c. Does fidelity of implementation of Student Success Skills impact student academic growth? d. Are the items loaded in the JMCI and the SESSS as previously reported in research? e. What is the result of factor analysis of the SESSS? Does that factor analysis support the results from previous research. This presentation provides attendees with a visual illustrating a useful process for assessing student needs, organizing intentional school counseling interventions, and evaluating the impact of school counseling interventions on student outcomes. Student outcomes can be measured using instruments or tracking student behavior change. In this presentation, the outcomes will be illustrated through the use of the JMCI and SESSS instruments, with correlation to academic critical data elements.

Expected Outcomes

Currently, statistical analysis is currently being done to produce outcomes related to the study. A synopsis of the major findings so far: 1) A significant increase in SESSS scores pre-post. 2) We do not see a significant increase in Junior Metacognitive Inventory scores pre-post. 3) In both cases there is a significant difference among schools. Schools have different baseline levels of skill use. 4) Re the SESSS, there is a significant school x time (pre-post) interaction. 4 schools show a significant pre-post increase in SESSS scores, one shows a significant decrease in SESSS scores. 5) CFA confirms the SESSS our previously determined factor structure. 6) EFA suggests that there are 2 factors for the JrMAI (as intended by the developers) but the developers item loadings do not exactly correspond to these factors. Five of the 18 items are reassigned to the other scale based on the EFA. A CFA confirms our EFA. We are now computing effect sizes for the SSS-related changes in SESSS scores and creating tables and figures for you report and for publications. We will also be replicating these analyses using SESSS and JrMAI sub scales (rather than total scale scores). Finally, we will be looking at the correlations of the SESSS sub scales with the JrMAI subclass.

References

Carey, J., Brigman, G., Webb, L., Villares, E., & Harrington, K. (2014). Development of an instrument to measure student use of academic success skills: An exploratory factor analysis. Measurement and Evaluation in Counseling and Development, 47, 171-180. Sperling, R.A., Howard, B.C., Miller, L.A., & Murphy, C. (2002). Measures of children's knowledge and regulation of cognition. Contemporary Educational Psychology, 27, 51-79.

Author Information

Brett Zyromski (presenting / submitting)
Northern Kentucky University
Counseling, Social Work, and Leadership
Highland Heights
University of Massachusetts, Amherst, United States of America
Northern Kentucky University
Center for Educator Excellence
Highland Heights

Update Modus of this Database

The current conference programme can be browsed in the conference management system (conftool) and, closer to the conference, in the conference app.
This database will be updated with the conference data after ECER. 

Search the ECER Programme

  • Search for keywords and phrases in "Text Search"
  • Restrict in which part of the abstracts to search in "Where to search"
  • Search for authors and in the respective field.
  • For planning your conference attendance, please use the conference app, which will be issued some weeks before the conference and the conference agenda provided in conftool.
  • If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.