Early Enrolment Predictors of First-Year Academic Performance of the Adult Learner in Higher Education

Author(s):

Sylvia Chong(presenting / submitting)

Conference:

ECER 2016

Network:

09. Assessment, Evaluation, Testing and Measurement

Format:

Paper

Session Information

09 SES 03 A, Research into the Predictive Validity of Individual and Contextual Characteristics for Academic Success and Returns on Education

Paper Session

Time:

2016-08-23

17:15-18:45

Room:

NM-F101

Chair:

Monica Rosén

Contribution

Higher education plays a key role in developing the competitive advantage for the society. Societies are increasingly dependent on skilled and talented workforce. Singapore is facing challenges of rapidly changing population. Society is aging at the same time that the birth rate is falling (National Population and Talent Divisiion (NPTD) 2013). A key pressure felt throughout the educational system is the increasing participation rate of non-traditional students. The fastest growing population in higher education is the working adult student (Finn, 2011).

In 2012, the Singapore government affirms its commitment to continuing higher education sector by growing the University pathways (Ministry of Education [MOE], 2012). The restructuring of higher education pathways is to support the changing needs of Singapore’s workforce. SIM University is Singapore's only privately funded university dedicated to working adults. The University’s pathways provided for many to pursue lifelong learning and higher education while balancing career, family and social responsibilities. How does SIM University inform and modify their strategies and processes to serve the diverse needs of today’s multigenerational students? This study was developed in the context of the SIM University’s enrolment process.

With influx of adult learners in higher education, it is crucial to develop an insight of their enrolment factors and patterns. Enrolment management and decisions often include anecdotal analysis. Understanding the potential academic performance of incoming applicants can support decisions. The University has a voluminous amount of data; however there is a need to convert the data into strategic knowledge for decision-making processes (Koh and Chong, 2014). Student offices are inundated with data from a variety of applications and sources, such as student demographics, professional experience and academic background. Data mining have emerged in higher education’s ability to capture a vast amount of data. Huebner (2013), defined educational data mining as “an emerging discipline that focuses on applying data mining tools and techniques to educationally related data” to “develop models for improving learning
experiences and improving institutional effectiveness”.

This study investigated the entering characteristics and early engagement variables that predict the success of first-year adult learner in a Singapore higher education institution. Data mining techniques are employed as part of the knowledge discovery process in this study. In the study first year academic performance is defined by cumulative grade point average. Through this, the University aims to identify students or applicants who are academically at-risk as early as possible. Decision trees and models are used to construct so that appropriate strategies could be designed and implemented. Specifically the study aimed to achieve the following

Identify characteristics that are available at enrolment variables of adult learners who are academically (CGPA) at-risk in higher education
Construct models for early prediction of the academically at-risk (Sem1 CGPA<2.3)
Evaluate these models using cross-validation

Making informed decisions require accurate data and analysis for evidenced-based insights and knowledge discovery. Building a predictive model to identify the potential at risk students or student success requires a combination of explicit knowledge base with sophisticated analytical approaches to uncover patterns, associations and relationships Modelling development is a reiterative stepwise process. Based on research objectives, the CHAID model was selected for optimum in complexity, accuracy, explanatory power over a range of variables.

The predictive model provided a consistent and equitable approach to enrolment management. Awareness of how potential students may perform academically could lead to a more targeted marketing campaign. Recruitment resources on academic, mentoring, and student success can aid in supporting the adult learners transition to University.

Method

Data mining, as a form of exploratory data analysis, is the process of extracting patterns and relationships from data rather than testing pre-formulated hypotheses. The methodology has a non-rigid sequence of four phases: 1. Pre-processing of data: explored data source and quality 2. Variable selection: identify the variables that were included for modelling. 3. Modelling: analytics integration towards building models. 4. Evaluation: Models are evaluated and validated. Pre-processing of Data The target sample for this study comprised first-year students who started their part-time degree program at SIM University in July 2013. Data were extracted from an in-house student information management system, which catalogues data from numerous sources within the admissions office as well as in other divisions of the university. Students with missing data were removed from the dataset to cater to some data mining algorithms which were not able to handle missing data. This resulted in a final sample of 2,392 students that were used for analysis. Data Selection Data selection of the variables for modelling in this study utilised a two-prong approach. Initial selection of variables that are admitted for the model is based on statistical importance with respect to the dependent variable based on correlation analysis and data mining algorithms. Variable selection is further enhanced via a literature-driven approach to improve the comprehensibility of variables. The literature-driven approach employed the use of domain knowledge to corroborate, validate and streamline the variables derived from the data-driven approach. Modelling Modelling development is a reiterative stepwise process. First steps involved the development of a reference decision tree with various algorithms (C5.0, CHAID and CRT) on the full dataset (2,392). A literature driven approach was used with alternative CHAID models. These alternative models were evaluated for predictive performance and/or that can better inform practices and policy development. Models were chosen based on a data-driven statistical approach as well as on domain knowledge that show an influence on students’ performance. Evaluation Lastly, the selected model is subjected to a cross validation process, with the goal of determining their predictive performance, preventing of over fitting and determining model stability. Subsequent CHAID model was used as a baseline model to build decision trees. In this phase, different statistically significant and contextually key factors were used as tree splitting criterions. The chosen model was then tested for its stability and replicability using a cross-validation method.

Expected Outcomes

The selected models were subjected to cross validation and were the ones with relevant explanatory logic and sound predictive performance with regards to the research objectives. Eventually, the CHAID model was chosen as the baseline reference model as it illustrated the inner workings of the model and its predictive performance is comparable with others. Out of the models, the Pre-University Institution model was selected as it offers a comparable specificity and sensitivity indices with no significant statistical trade. The chosen model was then tested for its stability and replicability using a cross-validation method. The result suggests a reasonably stable model and consistent predictive performance. From this final model, the University would be able to draw up targeted intervention and support strategies for adult learners in their initial years of study. In the selected model, the Pre-University institution, their pre-univiersty CGPA score and ‘O’ levels Engish and Mathematics grades emerged as important predictors of students’ academic performance for the first semester. The quality of learners’ academic foundation is intutively expected to impact on how students cope with the unviersty demands. The finding that the leaners’ pre-university institution is a key predictor indicates a wide variation in expections and standards of performance in the different pre-university institutions. The next finding that pre-university CGPA is a key predictor of academic outcomes is consistent with Geiser and Santelices (2007)’s study with 80,000 students that concluded that high school GPA is the strongest predictor of four-year college. This is being further elaborated with English and Mathematics skills being critical parts of the undergraduate tool kit for success.

References

Finn, D. (2011). Principles of adult learning: An ESL context. Journal of Adult Education, 40, 34–39. Huebner, R.(2013), “A.Educational data-mining research”, Research in Higher Edu. Journal, pp.1-13. Koh, H.C. and Chong, S. (2014) Broadening and Deepening of SoTL with Learning Analytics, Available: http://tlc.unisim.edu.sg/research/AdvSoTL/pdf/koh_hian_chye.pdf [20 June 2015]. Geiser, S. and Santelices, M.V. (2007) Validity of High-School Grades in Predicting Student Success Beyond The Freshman Year, Berkeley: Center for Studies in Higher Education: University of California. Ministry of Education (MOE). (2012). Report of the Committee on University Education pathways beyond 2015 (CUEP). Singapore: Author. Available: http://www.moe.gov.sg/media/press/files/2012/08/cuep-report-greaterdiversity-moreopportunities.pdf [20 Dec 2015]. National Population and Talent Divisiion (NPTD) (2013) A Sustainable Population for a Dynamic Singapore: Population White Paper, Available: http://www.nptd.gov.sg/content/NPTD/news/_jcr_content/par_content/download_98/file.res/population-white-paper.pdf [20 Dec 2015].

Author Information

Sylvia Chong (presenting / submitting)

SIM University, Singapore

Research

Singapore

Search the ECER Programme

Search for keywords and phrases in "Text Search"
Restrict in which part of the abstracts to search in "Where to search"
Search for authors and in the respective field.
For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.