The aim of this study is to examine the fairness of assessing learning journals both as the fairness in creating a valid and robust marking process as well as how different student groups may have unfair disadvantages in performing well in reflective assessment tasks. We investigate this fairness by looking at the challenges and requirements for reliably assessing a large number of learning journals, and by exploring possible pre-existing tendencies or settings towards reflective practice.

Our decision to include reflective journals as part of the assessment in the business ethics courses was informed by previous research which provided us with information on how to actively and explicitly promote critical reflection as part of pedagogical design (Carson and Fisher 2006; Cunliffe 2004; Kember et al. 1996), how to categorise levels of reflection (Mezirow 1990,1991; Barnett 1997; Carson and Fisher 2006; Kember et al. 2008, 1999); and the importance of reflection in tertiary business education (Reynolds 1998).

While we believed that (a) there should be the best possible alignment between the intended learning outcomes and forms of assessment and that (b) learning journals provide us with this important alignment, we struggled with the use of learning journals. Our challenges developed from two different perspectives: firstly, we have asked ourselves about the validity of judging and marking something as personal and unstructured as a reflective journal for assessment, and secondly, we were conscious of the possibility that certain student groups may be unduly disadvantaged by the introduction of reflective journals as an assessment tool.

Our questions were:

- How do our students approach reflection and are there contextual influences on how they reflect?

- What can be included in the pedagogical and assessment design to support student reflection?

- What is required to make assessing large number of learning journals by multiple assessors a reliable and valid process?

Our research approach is novel in the reflective journal literature in three key ways. First, we compare two large cohorts with several journal entries per student and second, those cohorts are from different cultures - Finland and Australia. Third, we take a dominantly quantitative approach to evaluating learning journals as assessment tools. This paper also includes our reflections on using a well-known reflective journal analysis framework by Kember et al (2008) as a basis for assessing and evaluating a large number of journal entries

Barnett, R. 1997. Higher education: A critical business. Buckingham, UK: The Society for Research into Higher Education and Open University Press. Carson, L., and K. Fisher. 2006. Raising the bar on criticality: Students’ critical reflection in an internship program. Journal of Management Education 30, no. 5: 700–23. Cunliffe, A. L. 2004. On becoming a critically reflexive practitioner. Journal of Management Education 28, no. 4: 407–26. Kember, D., A. Jones, A. Loke, J. McKay,, K. Sinclair,, H. Tse, C. Webb, F. Wong, M. Wong, P.W. Yan and E. Yeung. 1996. Developing curricula to encourage students to write reflective journals. Educational Action Research 4, no. 3: 329–48. Kember, D., A. Jones, A. Loke, J. McKay,, K. Sinclair,, H. Tse, C. Webb, F. Wong, M. Wong, and E. Yeung. 1999. Determining the level of reflective thinking from students' written journals using a coding scheme based on the work of Mezirow. International Journal of Lifelong Education 18, no. 1: 18–30. Kember, D., J. McKay, K. Sinclair, and F.K.Y. Wong. 2008. A four-category scheme for coding and assessing the level of reflection in written work. Assessment and Evaluation in Higher Education 33, no. 4: 369–79. Mezirow, J. 1990. How critical reflection triggers transformative learning. In J. Mezirow (& Associates): Fostering critical reflection in adulthood, 1–20. San Francisco: Jossey-Bass. Mezirow, J. 1991. Transformative dimensions of adult learning. San Francisco: Jossey-Bass. Reynolds, M. 1998. Reflection and critical reflection in management learning. Management Learning 29, no. 2: 183–200.