Attitudes Towards Online Multiple-Choice Assessment Tools: The Enthusiast, the Sceptic and the Unorthodox
Conference:
ECER 2008
Format:
Paper

Session Information

27 SES 04C, Didactical Use of ICT to Support Learning

Paper Session

Time:
2008-09-10
16:00-17:30
Room:
B3 333
Chair:
Ingrid Maria Carlgren

Contribution

Research topic This article discusses the use of an online multiple-choice tool in various educational settings. The discussion is based on case studies from three different faculties at a Scandinavian institution of higher education – the Faculty of Nursing, the Faculty of Teacher Education and the Faculty of Engineering. A common denominator of the three faculties is that they all use the same virtual learning environment (VLE) - a commercial piece of software that has been purchased and promoted by the central administration of the College. The main research questions addressed in this paper are as follows: - How do academics use online multiple-choice assessment tools within the realm of their didactical practice? - What is their general attitude towards multiple-choice assessment tools? - How does this attitude relate to the pedagogical discourse that is prevalent within their faculty or department? Theoretical framework The analysis of the case studies uses a theoretical framework that identifies several perspectives on learning, each of which is based on a different set of basic assumptions on what knowledge is and how it is acquired (Bruner 1996; Greeno et al. 1996; Greeno 1997). The designation of those perspectives varies across the literature, but it is possible to identify a number of major streams. - One of those streams would include the associationist/behaviourist perspective, which views knowledge as a set of specific behavioural responses to specific stimuli. Learning consists therefore of a acquiring the appropriate responses through repetition, facilitated by a programme of positive reinforcement (Skinner 1976). - Another stream would comprise the domain-structural or cognitive perspective, which considers learning to be about individual reasoning and understanding, as well as strategic problem solving (Piaget 1970). - A third stream would include perspectives referred to as socioconstructivist, sociocultural, socio-historical, or situative. Those perspectives portray learning as a social and cultural process, and emphasise the role played by the community or communities the learner belongs to (Lave and Wenger 1991).

Method

Methodology Three faculties were selected for the purpose of this study. In all three faculties studied, several members of the teaching staff were approached and interviewed about a number of aspects of their teaching practice, with a particular focus on their use of the standard VLE solution available throughout the institution. The study uses a broadly interpretative methodological approach, including the gathering of qualitative data from a variety of sources and a data analysis method inspired by actor network theory (ANT) (Callon 1986; Latour 1987; Latour 1988; Akrich 1992; Latour 1999). The empirical data that forms the basis of the case studies was collected primarily via face-to-face interviews and personal logs (diaries) kept by the respondents, typically over a period of one week, providing details of their VLE-based activities and their rationale. The data was then analysed using key ANT conceptual tools such as networking, inscription, translation and punctualization.

Expected Outcomes

Findings The data from the case studies reveals that both the attitude of the teaching staff towards online multiple-choice tools and their actual practice vary greatly from one academic milieu to another. The paper investigates more particularly the relation between staff attitude to multiple-choice tools and the underlying pedagogical philosophy permeating the teaching environment they belong to. Case 1 The paper reveals that the respondents from the Faculty of Engineering consider both a cognitive and a sociocultural learning philosophy to be legitimate approaches to learning. This conviction makes the online multiple-choice tool a natural didactic practice when selecting a method to assess the students’ learning with regards to areas of knowledge where answers to questions can easily be categorized as “right” or “wrong”. Case 2 At the Faculty of Education, however, the attitude among academic staff towards multiple-choice tools is quite different. The general standpoint on multiple-choice assessment methods is that they participate to a cognitive/behaviourist approach to learning against which the Faculty takes a strong stance. Assessment forms based on a multiple-choice format are generally considered incompatible with the widely embraced sociocultural philosophical approach of the Faculty and are therefore almost unanimously rejected by the respondents. Case 3 The data gathered from the Faculty of Nursing point towards a more unorthodox pattern of use of the multiple-choice tool. Some of the staff members at that faculty have chosen to let students create multiple-choice questions themselves and submit those questions to their fellow students. The students are encouraged to discuss among each other the choice and formulation of the questions and of the proposed answers. Although the academic staff members do not partake in any direct quality check of the questions and answers, they are actively involved in the process of facilitating the students’ discussions about how to identify the important parts of the curriculum, how to assess them and what criteria should be used in the assessment process. This case study reveals that didactic practices that use multiple-choice tools extensively may help academic staff broadening the repertoire of assessment methods available within the realm of a sociocultural approach to learning. Conclusion The study reveals that patterns of use of the multiple-choice tool under investigation are highly influenced by the underlying teaching philosophy that is prevalent in the respondents’ academic environment. In two of the case studies, the tool is considered to be an instrument best suited for a cognitive/behaviourist type of assessment and is therefore either adopted or rejected on the basis of that very “image”. In the third case study, the respondents do not express any particular preconception about the underlying pedagogical perspective of multiple-choice tools. This lack of preconception may explain why the tool ends up being “twisted” so as to fit into a sociocultural philosophy of teaching, and thereby acquires a new status as an integral part of the assortment of assessment tools considered appropriate for sociocultural learning. Further research may involve investigating the use of multiple-choice tools in different contexts, for example in other faculties, in other educational institutions, or within the realm of workplace-based learning and information sharing. A deeper examination of the learning implications of using “untraditional” multiple-choice assessment forms may also be an interesting focus for further investigation of the issues raised in this paper.

References

References Akrich, M. (1992). The de-scription of technical objects. Shaping technology / building society: studies in sociotechnical change. W. E. Bijker and J. Law. Cambridge Ma, The MIT Press: 205-224. Bruner, J. S. (1996). The Culture of Education. Cambridge, Mass., Harvard University Press. Callon, M. (1986). Some elements of a sociology of translation: domestication of the scallops and the fishermen of St Brieuc Bay. Power, Action and Belief. A New Sociology of Knowledge? J. Law. London, Routledge & Keagan. Sociological Review Monograph 32. Greeno, J. G. (1997). "Theories and practices of thinking and learning to think." American Journal of Education 106(1 Nov 1997): 85-126. Greeno, J. G., A. M. Collins, and Resnik, L. B. (1996). Cognition and learning. . Handbook of Educational Psychology. R. C. Calfee and D. C. Berliner. New York, Simon & Shuster Macmillan: 15–46. Latour, B. (1987). Science in Action. Cambridge, MA, Harvard University Press. Latour, B. (1988). The Pasteurization of France. Cambridge, Mass and London, England, Harvard University Press. Latour, B. (1999). On recalling ANT. Actor Network Theory and After. J. Law and J. Hassard, Blackwell: 15-25. Lave, J. and E. Wenger (1991). Situated Learning: Legitimate Peripheral Participation. Cambridge, Cambridge UP. Piaget, J. (1970). Science of Education and the Psychology of the Child. New York, Orion Press.

Author Information

Faculty of Education, Oslo University College, Norway
Oslo University College
Faculty of Education
Oslo
158
Centre for Educational Research and Development, Oslo University College, Norway

Update Modus of this Database

The current conference programme can be browsed in the conference management system (conftool) and, closer to the conference, in the conference app.
This database will be updated with the conference data after ECER. 

Search the ECER Programme

  • Search for keywords and phrases in "Text Search"
  • Restrict in which part of the abstracts to search in "Where to search"
  • Search for authors and in the respective field.
  • For planning your conference attendance, please use the conference app, which will be issued some weeks before the conference and the conference agenda provided in conftool.
  • If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.