Session Information
Paper Session
Contribution
The central theme of this paper focuses on the recent shift from standardization to differentiation in assessments through the advent of digitalization in higher education. Digitalisation has been borne as an artificial solution to academia’s issues with the contextual complexity of assessment. It addresses the issues surrounding a reluctance of some academics to mark and grade assignments for benchmarking, tracking, and scaffolding which have been swiftly overshadowed by digitalization, and particularly during the recent Covid-19 global pandemic. Elements of digital transition include Artificial Intelligence (AI) software applications, which have been able to inspire an entire new learning community in preparing graduates with new and desirable skills, knowledge, and ecumene with the aim of prospering in the world of digitalized futures.
This inquiry into digitalization has allowed us to explore tangible evidence of what is learnt in an equitable fashion, the breadth and depth of disciplinary knowledge, and how this knowledge is conceived, internalized, reflected on, expanded, and abstracted. Parenthetically, this combines linkages between theory / knowledge, disciplinary methodological approaches, and prospects for further application in the real world outside the university.
As a caveat, it is our understanding that technology continues to advance the workforce and, as such, so too should the advancement of university teaching and learning. Higher education institutions have responded by re-evaluating outdated teaching and learning methods and embracing new technologies in approaches ranging from establishing learning management systems to shifting from teacher-focused to student-centred learning. Unfortunately, much of the technology advancements to date and particularly as a function of the pandemic, have focused merely on online delivery---relying heavily on learning management systems---and much less on the technical utility and applicability of teaching and learning analytics.
Inspired by digitalization issues in the provision of the quality of teaching and learning in higher education, efforts to uncover an alignment between academic teaching and assessment have attracted our attention to trial what personalized assessments may accomplish across academic discipline and cultural and learning boundaries.
A digital Personalised Assessment Tool (PAT) was developed initially in Australia in response to Gonski et. al's white paper (2011) on the quality of education. The PAT is an attempt to provide a more equitable assessment solution in analyzing and recognizing learner difference among undergraduates and postgraduates from diverse learning backgrounds (urban, regional, rural, remote). Developed as an alternative to standard assessment tools and utilizing a theoretical modification of Seligman's 2002 strength-based assessment approach, the PAT was further enhanced for exploring how five higher order skills (creativity, communication, analytical thinking, problem solving, and reflective thinking) might be better benchmarked, tracked, and scaffolded using Fiske and Taylor's social cognition theory (2013). The PAT was subsequently trialed in China and Sweden to explore the applicability of teaching and learning analytics across cultures, institutions, and disciplinary boundaries.
Method
In the current paper, we present on a comparative case study analysis that was undertaken to systematically describe cultural, institutional, and disciplinary contexts where the PAT was piloted (Australia, China, and Sweden). The aim of the comparison was to determine the preparedness of academic staff to utilize the PAT, the identification of inferences drawn from using the PAT by various academic staff, and the utilization of the instructional data for individual and institutional gain. The objective of the study was to determine if the PAT was viewed as equitable, technologically relevant, and whether it promoted 'quality' assessment. The research questions explored: 1) Can alternative assessments like PAT be used to provide a more equitable approach to promote differentiation while, at the same time, adhere to core disciplinary requirements? 2) Is it possible to create a bottom-top approach towards implementing new technologies in academic assessments? 3) Could the benchmarking, tracking and scaffolding of salient higher order skills improve teaching and learning as well as feedback? The current presentation reports on case studies that were collected from 2016-2023, which include over 150 undergraduates and postgraduates who volunteered to be assessed using the PAT along with academic staff approvals. Ethical practices were adhered to, including GDPR policy guidelines, with emphasis placed on curriculum development by each respective higher education institution.
Expected Outcomes
While the comparative cases are exploratory in nature, the paper presents on empirical evidence in assessing each student (n=150) on five higher order skills and representing over 20 psychometric characteristics, in order to highlight how a systematic approach toward assessment invites a change in academic discourse toward quality assessment over time. The findings yield indications regardless of culture or institution. Approaches toward the utility of PAT proved meaningful. These seemed to motivate a renewed interests in showcasing individual student's ability and learning capability. These revealed the function of a form of gatekeeping for admissions purposes on the one hand and enabling at-risk students to complete their studies in the other. Presenting the data in the form of assessment visualisations further encouraged a non-apprehensive dialogue to take place between academic staff and students. Such dialogue subsequently improved assessment feedback practices and gave further carriage that allowed to delve deeper and more broadly into core curricular content. The paper presentation elaborates and illustrates the details of these findings and invites input and critical comments from the community of scholars attending the paper session.
References
Denman, Brian D. & Meeri Hellstén (in press). Equity and Quality Education: Retrospect and prospect. International Handbook on Education Development in Asia Pacific. Wing On Lee, Phillip Brown. A. Lin Goodwin and Andy Green, eds. Springer Press. Denman, Brian D., (2022). "From Global Thinker to Innovative Mind". In: International Handbook of Education Development in Asia Pacific. Wing On Lee, Phillip Brown, A. Lin Goodwin, and Andy Green, eds. Springer Press, https://link.springer.com/referenceworkentry/10.1007/978-981-16-2327-1_18-1 Fiske, Susan T., and Shelley E. Taylor (2013). Social Cognition from Brains to Culture, 2nd Edition. Sage, London. Gonski, D., Boston, K., Greiner, K., Lawrence, C., Scales, B., & Tannock, P. (2011). Australian government review of funding for schooling: Final report December 2011. Department of Education EaWR (Ed). Canberra City, ACT: Department of Education, Employment and Workplace Relations.
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.