A number of preliminary researches in different countries (Mitana at al., 2019; Deming, 2017; Kautz et al., 2017; Duncan & Dunifon, 2012; Heckman & Kautz, 2012) highlighted the current critical labour market soft skills needs and examined the levels and acquisition of High Order Thinking (HOT) and soft skills in lower secondary schools. These studies reveal that, whereas some employers are satisfied with the technical skills possessed by entry level employees, they are not satisfied with their level of soft skills. 

 In these regards, most assessments and evaluations of Uganda’s secondary education do not address soft skills development. They instead emphasise cognitive skills through standardised examinations and tests scores that concentrate on students’ mastery of content knowledge that has been traditionally examined through public examinations and assessments. 

 Although the OECD has led key international studies on education and skills for many years including Programme for International Student Assessment (PISA), The Programme for the International Assessment of Adult Competencies (PIAAC), on literacy, numeracy and problem solving and The Teacher and Learning International Survey (TALIS), and literature has evidenced that soft skills clearly affect academic performance and life after school, policy makers and educators have not leveraged that fact (Farrington et al., 2012).

 Not assessing soft skills has presented a dichotomy between the general aims of education, in Uganda and also in Europe, and what is assessed in schools; and more importantly, between what is assessed and what is required of a student to cope with life after secondary school. 

 In this research, Soft skills “refer to a broad set of skills, competencies, behaviours, attitudes, and personal qualities that enable people to effectively navigate their environment, work well with others, perform well, and achieve their goals” (Lippman et al., 2015 pp 4). The approach to choose the key soft skills and to elaborate the theoretical support was The Big Five Model (Goldberg, 1993). 

 The objective of this study is double. Firstly, to provide a psychometric evaluation of the quality of the High Order Thinking and Soft Skills Questionnaire (HOT-SOKS-Q). Secondly, to analyse the relationship between HOT, Soft-skills and the scores in literacy and numeracy, as learning results. A reliable assessment of soft skills, as well as their relationship with academic performance, will help school leaders and teachers to include them in the teaching-learning process. In addition, these results will offer relevant information for policy makers and other stakeholders. 

 The Preliminary study akin to needs analysisinvolved qualitative interviews of respondents representing diverse industry classification. These structured interviews generated information on thematic areas of soft skills that are of respondents’ current and future need. The approach used to develop HOT-SOK-Q followed a paradigm that prizes demand driven supply of trainees. HOT and Soft skills domains that were identified at the preliminary stage include 11 constructs: problem solving, critical thinking, responsibility, achievement striving, grit, integrity/honesty, assertiveness, teamwork, compassion/empathy, self-control, and self-esteem. 

 At the pilot stage, 79 items that were hypothesized to be explained by the 11 constructs were tried on 380 students with a 1-5 Likert score space. Psychometric evaluation of the soft skills tool through the pilot data identified 8 items that were not satisfactorily explained by the constructs and were dropped from the test tool.

 The test phase which is the focus of this research involved the administration and psychometric analysis of the 71 items tool with a 1-3 Likert scale, and Literacy and Numeracy tests.

 Based on the findings from this phase, the tool is being piloted with a nationally representativesample of 2064 students of senior 3

The tool being evaluated had 71 items which were designed with a 1-3 Likert score space. Five-hundred-thirty (530) secondary school students of Senior 3 (S3) responded to the items by means of self-reporting. Responding students were from five different schools in Kampala, Wakiso, Pallisa and Pader districts. Literacy and Numeracy test had 13 and 10 constructed-response items. Standardized Structural Equation Models (SEM) were used to examine how the 11 latent domains of soft skills are associated with the proposed test items. In other words, SEM was used to test the unidimensionality of the hypothesized indicators of the construct. Model fit was evaluated using four indices: Root Mean Square Error Approximation (RMSEA) - is significant if less than 0.05; Likelihood Ratio Chi-Squared shows significant fit if p – value is greater than 0.05; Comparative Fit Index (CFI) demonstrates good fit if close to 1; and Tucker-Lewis Index (TLI) shows significant model fit if close to 1. Appropriate modification indices were implemented where applicable. To assess the quality (reliability and validity) of the soft skills tool and the achievement tests, Rasch partial credit model (Wilson, 2004) which is an Item Response Theory (IRT) estimator was used to compute test items and person characteristics. The above model is assumed as the Item Response Function (IRF) because it meets the invariance principle, that the order of the item responses must remain the same for all respondents and the order of the respondents must remain the same for all item responses (Wilson, 2004). Wright Maps will be constructed based on person and item parameters to help validate Internal Items Structure as were hypothesized, i.e., Item order and Construct order. Reliability was empirically estimated using Cronbach’s and Person Separation Reliability coefficient. The model choice was evaluated by establishing whether items and person fits the model. Item fit was examined using the weighted statistic the Infit Mean Square Error (infitMNSQ) and a weighted t statistic. Researchers (Wilson, 2004) have indicated an acceptable range for the infitMNSQ: that is, a value outside the bounds accompanied by corresponding t statistics is an indication of item misfit. Rasch Partial Credit Model (RPCM) was assumed and estimated using the crasch psychometric package in the R environment. Data fit to the RPCM was evaluated in contrast to other unidimensional as well as multidimensional models using the mirt package in R environment.

Regarding the psychometric properties of the HOT-SOK-Q, the 11 constructs were demonstrated to be unidimensional using SEM; In other words, all the constructs fitted the model very well (Problem Solving, Grit, Compassionate/empathy, cooperation/teamwork, self-control/patience), outstanding well (Assertiveness, Self-stem), exceedingly well (Critical thinking, Responsibility, Achievement striving) or exceptionally well (Integrity/honesty). Indeed, all the domains of the HOT and soft skills revealed valid Internal Structure of items. And all the items were found to be significantly associated with the constructs except one item for Grit construct and one for cooperation/teamwork. In presenting the findings regarding reliability of the domains, we are cognisant of their nature of being socially acceptable. For Problem Solving, Critical Thinking, Cooperation/teamwork, the domains had 5,5,8 items respectively and satisfactorily separate the respondents, that is satisfactorily reliable. For Responsibility, Achievement Striving, the domains had 6,5 items and reliably measure the constructs, that is satisfactorily reliable. For Self-control, Self-esteem, the domains had 7, 5 items, respectively and quite satisfactorily separate the respondents. Thus, quite reliable measures. For Grit, the domain had 8 items, and unsatisfactorily separate the respondents. That is, the test items don’t reliably measure the construct. For Integrity/Honesty, Assertiveness, the domains had 5, 10 items, respectively and for the first domain, the items quite satisfactorily separate the respondents. For Compassionate/empathy, the domain had 4 items and, poorly separate the respondents. Overall, for the whole scale, all the HOT and soft skills items fit the Partial Credit model. Lastly, regarding the relationship between the soft-skills scores and literacy and numeracy achievement, the results show that although the magnitude of the linear association coefficient are small, they are positive and statistically significant. However, additional information could be sought from the association of each of the 11 domains with either literacy or numeracy

Deming, D. J. (2017). The Growing Importance of Social Skills in the Labor Market. The Quarterly Journal of Economics, 132, (4), 1593–1640. Duncan, G. J. & Dunifon, R. (2012), “Soft-Skills” and Long-Run Labor Market Success, in Solomon W. Polachek, Konstantinos Tatsiramos (ed.) 35th Anniversary Retrospective (Research in Labor Economics, Volume 35) Emerald Group Publishing Limited, pp.313 – 339. Farrington, C. A., Roderick, M., Allensworth, E., Nagaoka, J., Keyes, T. S., Johnson, D. W., & Beechum, N. O. (2012). Teaching adolescents to become learners. The role of noncognitive factors in shaping school performance: A critical literature review. Chicago: University of Chicago Consortium on Chicago School Research. Goldberg, L.R. (1993). The structure of phenotypic personality traits. The American Psychologist. 48 (1): 26–34. Heckman, J. J. & Kautz, T. (2012). Hard evidence of soft skills. Labour Economics, 19 (4), 451-464. Kautz, T., Heckman, J. J., Diris, R., Weel, B., & Borghans, L. (2017). Fostering and measuring skills: Improving cognitive and non-cognitive skills to promote lifetime success. Chicago: Directorate for Education and Skills Centre for Educational Research and Innovation (CERI). Lippman, L.H., Ryberg, R., Carney, R. & Moore, K.A. (2015). Key “Soft Skills” that Foster Youth Workforce Success: Toward a Consensus Across Fields. Washington, DC: USAID, FHI 360, Child Trends. Published through the Workforce Connections project managed by FHI 360 and funded by USAID. OECD (2017). Better understanding our youth’ social and emotional development. http://www.oecd.org/education/ceri/Better%20understanding%20our%20youth%27s%20social%20and%20emotional%20development%202018.pdf Wilson, M. (2004). Constructing measures: An item response modeling approach. Routledge.