Development And Validation Of An Instrument To Determine Trainee Computer Science Teachers’ Perceptions Of Computational Thinking

Author(s):

Andrew Csizmadia(presenting / submitting)

Conference:

ECER 2015

Network:

Emerging Researchers' Group (for presentation at Emerging Researchers' Conference)

Format:

Paper

Session Information

ERG SES H 11, Teachers and Education

Paper Session

Time:

2015-09-08

11:00-12:30

Room:

336. [Main]

Chair:

Fiona Chambers

Contribution

Computational Thinking is regarded as a gift from Computer Science to other discipline areas, a digital literacy skill and a cornerstone of the Computing Programme of Study to be delivered as part of the National Curriculum in English Schools. In this paper, the researcher describes the design, iterative development and validation of a qualitative participatory data collection instrument based on open card sorting. This is a new application of card sorting as a knowledge elicitation technique and its ability to elicit semi-tacit knowledge applied to Computer Science Education, where the subjects, trainee Computer Science teachers, are given a collection of cards containing Computational Thinking vocabulary to sort. They are asked to separate these cards into groups determined by the subjects themselves. Information about the individual subject’s knowledge structure of and their conceptual model of Computational Thinking is inferred from the groups created and the names attributed to these groups.

This paper discusses the challenges associated with analysing data generated by card sorts either semantically or syntactically. An open card sort activity is proposed as the most appropriate approach to be adopted for both knowledge acquisition and knowledge elicitation in order to explore the subjects’ knowledge of Computational Thinking vocabulary and their clustering of individual terms. In addition, subjects’ voice would be captured using a self-reflective constructive questionnaire in order to offer suggestions to both refine and enhance the next iteration of this research instrument. Finally, in section 4, conclusions and a discussion on the next stage of the development are presented.

Method

Card sort is participatory knowledge elicitations technique (Martin & Hannington 2012) in which participants are given a collection of items and are asked to organise and classify them into logical groupings. Information about the implied knowledge structure that participants construct is inferred from the groups created and the names used to describe those groups. The focus of this study is to determine whether a card sort activity is a validate instrument for eliciting perceptions of subjects. The subjects categorised and created self-deterministic groups based on the items that have been supplied to them and the names subjects gave to the resulting categories. Hudson (2012) indicated that the sorting process to be performed by participants in a card sort will provide the following information: Terminology (what subjects call things); Relationships (proximity, similarity) and Categories (groups and their names). The objective of this study was to discover if an open card sort knowledge acquisition instrument could be used to elicit how trainee Computer Science teachers organise, structure and articulate their knowledge and understanding regarding Computational Thinking terminology. An extensive literature review was conducted identifying 15 academic texts and 3 academic texts relating to Computation Thinking. Context analysis (Krippendroff 2013) of the identified texts indicated 100 terms to be used for the card sort. Terms identified were compared and contrasted identifying superficial similarities which could produce a superficial match and potentially generate unhelpful results (Hudson 2012).Superficial similarities were addressed by either removing one of the similar terms or if possible replacing one of the terms with a synonym. An open card sort was chosen to elicit and explore individual participant’s conceptual models for computational thinking. Participants were provided with a set of items to sort, collate into groups and then determine labels for those groups they created. An outcome of this study was to elicit conceptual model of Computational Thinking of individual participants. Hudson (2012) indicated that participants should be representative of the target group as they will be self-motivated to participate in the study because they potentially have an interest in the outcome of the study. Thirteen trainee Computer Science teachers were recruited as participants for this study, as they have a professional interest in understanding what Computational Thinking is.

Expected Outcomes

An effective and easy to implement open physical card sort activity has been created with clear guidance provided for subjects as to how they are to undertake the card sort exercise; Subjects performed sorts on items that could be categorised into a small number of well-defined categories; The card sort provided an effective research instrument of eliciting subjects’ knowledge structures of Computational Thinking terminology (Murphy et al. 2005); The card sort, as a by-product, provided individual subjects with an indication of potential deficiencies in their specialist computer science subject knowledge and understanding with regard to terminology associated with Computational Thinking (Murphy et al. 2005). The following questions have been asked either by the subjects or the researcher as they critically reflected on the card sort activity: Should a digital card sort application be developed? Could a digital card sort application support multi-categorisation of individual items? Should subjects repeat this activity using an open card approach, in which subjects are given the group headings and categorise items using the pre-defined headings? At present subjects conduct this card sort activity individually, should this be conducted as a small group activity? Should subjects use the Think Aloud technique to articulate their rationale for placing cards within groups?

References

Barr, V. and Stephenson, C., 2011. Bringing computational thinking to K-12: What is involved and what is the role of the computer science education community? ACM Inroads, 2(1), pp. 48-54. Brennon, K. and Resnick, M., 2012. New frameworks for the studying and assessing the development of computational thinking [online]. Available at: http://web.media.mit.edu/~kbrennan/files/Brennan_Resnick_AERA2012_CT.pdf [Accessed: 20th December 2013]. Coxon, A. and MacMillan, P., 1999. Sorting Data: Collection and Analysis. Thousand Oaks, CA: Sage Publications. Dede, C., Mishra, P., & Voogt, J., 2013. Working Group 6: Advancing computational thinking in 21st century learning [online]. Available at: http://www.edusummit.nl/fileadmin/contentelementen/kennisnet/EDUSummIT/Documenten/2013/Advancing_computational_thinking_in_21st_century_learning.pdf [Accessed: 20 January 2014]. DfE [Department for Education], 2013. National Curriculum in England: Computing programmes of study - key stages 3 and 4 [online]. Available at: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/239067/SECONDARY_national_curriculum_-_Computing.pdf [Accessed: 20 December 2013]. Gaffney, G., 2000. “What is Card Sorting?” Information & Design [online]. Available at: http://infodesign.com.au/usabilityresources/cardsorting/ [Accessed: 10 September 2014]. Hudson, W., 2012. Card Sort in The Encyclopedia of Human Computer Interaction, 2nd [online]. Available at: http://www.interaction-design.org/encyclopedia/card_sorting.html [Accessed: 20th December 2013]. Neilsen, J., 2004. Card Sorting: How Many Users to Test [online]. Available at: http://www.nngroup.com/articles/card-sorting-how-many-users-to-test/ [Accessed: 10 September 2014]. Neilsen, J., 2009. Card Sorting: Pushing Users Beyond Terminology Matches [online]. Available at: http://www.nngroup.com/articles/card-sorting-terminology-matches/ [Accessed: 10 September 2014]. Sanders, K., 2005. A multi-institutional, multi-international study of programming concepts using card sort data. Expert Systems, 22(3), pp.121-128. Schön, D., 1983. The reflective practitioner: how professionals think in action. New York: Basic Books. Spence, D., 2009. Card Sorting: Designing Usable Categories. New York: Rosenfeld Media. Tullis, T. and Wood, L.E., 2005. How Can You Do a Card-sorting Study with LOTS of Cards? Poster presented at the Annual Meeting of the Usability Professionals Association, June 24-July 1, Montreal, QB, Canada. Upchurch, L., Rugg. G. and Kitchenham, B., 2001. Using card sort to elicit web page quality attributes, IEEE Software, 18(4), pp. 84-89. Wing, J., 2006. Computational Thinking. Communications of the ACM, 49(3), pp. 33–36.

Author Information

Andrew Csizmadia (presenting / submitting)

Newman University

Education

Birmingham

Search the ECER Programme

Search for keywords and phrases in "Text Search"
Restrict in which part of the abstracts to search in "Where to search"
Search for authors and in the respective field.
For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.