This paper presents preliminary results of a first exploratory study, with multidisciplinary (Psychology, Economics, Market Law, Art History and Teacher Training) and multilevel (various undergraduate and master courses) conditions. This multidisciplinary glance is of utmost importance in Higher Education (Alexander et al., 2011). Up to now, we have plenty of educational research on educational students. Research on psychology-students or teacher-students is insufficient to understand learning processes at all areas of HE, and certainly HE-instructors at other disciplines would benefit from inquiry on their own students’ behaviors and motivation. The time has arrived to move beyond our lecture room walls and inquiry how students at other faculties tackle their learning processes and cope with the challenges (Quinlan, 2015). In this study, two different Spanish universities participate.

We designed a concrete instructional plan which offers a systematic study support for students with the purpose of enhancing self-regulated learning. We applied a particular psychometric algorithm on a system of learning tests, specially designed for promoting metacognitive engagement (Bruttomesso et al., 2003; Leclercq, 1993). This algorithm does not only evaluate the correctness of students’ responses to multiple-choice items, but also the degree of certainty of their given response. Other authors call it certainty-based-marking (CBM) (Gardner-Medwin, 2008). CBM breaks the traditional marking scheme (in our country 0-10), since grades are adjusted to the degree of confidence or certainty declared by the student (low-middle-high). For example, a 10-item test –as we used in our study- generates a grading range from -60 to +30. That is: students need to learn how to reinterpret their own results and make sense of them. All sorts of emotional, motivational and metacognitive reactions happen (Remesal et al. 2022a) when using this strategy. In this paper we want to focus on the students’ very first reactions to that new evaluative algorithm, within an instructional plan where this testing system has an underlined formative purpose. We look at CBM-results in connection with emotional reactions (positive and negative / activating and de-activating emotions, following Pekrun (2006); calibration (relation between expectations and achievement (Dinsmore & Parkinson, 2013; Hadwin & Webster, 2013) and metacognitive thoughts. All three phenomena together interweave towards new possibilities of self-regulated learning behavior (or lack thereof!) (Barr & Burke, 2013, Remesal et al. 2022b).

The instructional system we designed and put to the test roots on a view of self-assessment as the basic tool for self-regulated learning (Panadero et al. 2016). In this study we want to evaluate the effectiveness of such instructional system to pedagogically support complex learning processes of students at different disciplinary areas and levels (bachelor – masters).

The study is based on a mixed method strategy. A large sample of students from the different participating centers and courses (n= 1,526) take part by responding to a series of learning tests specifically designed to accompany self-regulated study during a semester. The first three tests -related to ad hoc contents of each specific course- referred to progressive thematic units of the syllabus of each of the subjects. These three learning tests, are strategically placed along the academic term, to facilitate students’ metacognitive activation, as an opportunity for progressive diagnostic self-assessment. The last of the tests took up all the previous questions, as a self-assessment closure. That is, we generated a sequential database with four learning data points, allowing the contrast between the beginning and the end of the learning process, as well as its evolution. The learning tests used contain 10 multiple-choice questions with 4 answer options and were designed by the teacher responsible for each subject. The marking algorithm produces a range of scores from -60 to +30 points. Immediately after responding, the student receives automatic feedback with the grade and their given answers (whether right or wrong). Also, students received a special guide for interpreting their results within a quasi-quartile scheme: negative range (-60 to 0 points), first positive range (1-10), second positive range (11-20), third positive range (21-30). These grades had no certification effect in the courses, but a pure diagnostic and formative aim as a way to prevent negative reactions and emotional and/or cognitive blockages in students. After each of the answers to the learning test, students answer - voluntarily, without implications for the academic course - a questionnaire of reflection and evaluation of the experience, where emotions and calibration are gathered. Finally, a small selection of students participated in an individual interview. In this paper we want to share results of the very first CBM-experience of all the participating students concerning differences at: • Emotions: retrospective, in reaction to the experience, and prospective, in advancing the subsequent learning experiences in the course. • Calibration: under-calibration, adequate calibration, and over-calibration; In addition to the variables indicated for area and level of study, the following demographic variables are also considered: sex, age, family burden (having children or other relatives in care), formal workload (no work besides studies, half-day job, full-time job), with the understanding that the last two may affect the time available for personal study.

At this time, data collection is still in progress and only preliminary results can be shown for those courses already closed during the first semester of the current academic year, referring to the area of teacher training in primary education (undergraduate) and secondary education (master's degree). Data from the second semester are still pending and will complete the contrast of disciplines (Psychology, Economics, Market Law, History of Arts) and the rest of the variables. Up to this moment we can report about a sample of 356 students, with a mean age of 25 years (S=6 years) with a range between 17 and 52 years. 64% are female, 36% male. Previous studies before accessing the current studies are: vocational education (4%), baccalaureate (10%), undergraduate (61%), master's degree (23%) and doctorate (2%). Forty-four percent do not work, 35% work part-time and 21% combine their studies with full-time work. Finally, 88% do not have family-care responsibilities, compared to 12% who do. The first results for this subsample concerning emotional reactions show significant differences in the emotional experience -but varying effect size-, both when reporting retroactive -positive and negative- emotions (How do I feel about my results?: joy, pride, relief / sadness, shame, anger) (Phi = 0.184) and proactive -positive and negative- emotions (How do I feel when thinking about tackling the rest of the semester?: expectation, hope / fear, uneasiness, boredom, indifference) (Phi = 0.556). Thus, positive emotions in reaction to this first encounter with CBM testing are less strong than instructors would desire. Nevertheless, facing the new learning challenges more positive than negative prospective emotions grow. Currently, we are expecting for the data collection phase to be completed during the second semester of this course, so that full final results can be offered at the conference.

Alexander, P. A., Dinsmore, D. L., Parkinson, M. M., & Winters, F. I. (2011). Self-regulated learning in academic domains. In B. J. Zimmerman, & D. Schunk (Eds.), Handbook of self-regulation of learning and performance. New York: Routledge. Barr, D. A., & Burke, J. R. (2013). Using confidence-based marking in a laboratory setting: A tool for student self-assessment and learning. Journal of Chiropractic Education, 27(1), 21-26. Bruttomesso, D., Gagnayre, R., Leclercq, D., Crazzolara, D., Busata, E., d’Ivernois, J. F., Casigila, E., Tiengo, A., & Baritussio, A. (2003). The use of degrees of certainty to evaluate knowledge. Patient Education and Counseling, 51(1), 29-37. Dinsmore, D.L. & Parkinson, M.M. (2013). What are confidence judgements made of? Students’ explanations for their confidence ratings and what that means for calibration. Learning and Instruction, 24, 4-14. Gardner-Medwin, A. (2008). Certainty-Based Marking: rewarding good judgment of what is or is not reliable. In: (Proceedings) Innovation 2008: The Real and the Ideal. London. Hadwin, A.F. & Webster, E.A. (2013). Calibration in goal setting: examination the nature of judgements of confidence. Learning and Instruction, 24, 37-47. Leclercq, D. (1993). Validity, reliability, and acuity of self-assessment in educational testing. In Item banking: Interactive testing and self-assessment (pp. 114-131). Springer Berlin Heidelberg. Panadero, E., Brown, G., & Strijbos, J-W. (2016). The future of Student self-assessment: a review of known unknowns and potential directions. Educational Psychology Review (28) 803-830. Pekrun, R. (2006). The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and practice. Educational psychology review, 18, 315-341. Quinlan, K. M. (2015) ‘Adding feeling to discourses of teaching and learning in higher education’, Asian Journal of the Scholarship of Teaching and Learning, pp. 5-8. Remesal, A., Álvarez-Brinquis, M., Carbó, M., El-Khayat, M., Fierro, J.D., Garcia-Mila, M., Gri, T., Jarque, M.J., Pérez-Clemente, G., Pérez-Sedano, E., & Vega, F. (2022a). Challenging the traditional grading scheme for metacognitive engagement at teacher education. Poster presented at SIG1+4. Cádiz 27-30/6-2022. Remesal, A.; Pérez-Sedano, E.; El-Khayat, M.; Fierro, J.D. (2022b). Fostering metacognitive engagement with CBM for competence-based programs. Online paper presented at SIG16-Metacognition-2022. Frankfurt.