Can Creativity and Critical Thinking be assessed as general constructs or as subject-specific skills?

Author(s):

Ourania Maria Ventista(presenting / submitting)Robert Coe(presenting)

Conference:

ECER 2015

Network:

09. Assessment, Evaluation, Testing and Measurement

Format:

Paper

Session Information

09 SES 07 D, Developing and Validating Instruments for Tests and Assessments

Paper Session

Time:

2015-09-09

17:15-18:45

Room:

395. [Main]

Chair:

Petr Soukup

Contribution

The research question is whether creativity and critical thinking are feasible to be measured with the existing measurement tools or these tools can help drawing solely vague conclusions about the evaluating outcomes. The purpose of the research was to pilot measurement tools of the existing bibliography which are supposed to assess critical thinking and creativity and to evaluate a measurement tool for creativity which has been created and used for the first time for this research.

Creativity is usually perceived as a broad term which includes a lot of sub-characteristics such as divergent thinking, convergent thinking, openness to explore new ideas and listening to “inner voice”(Treffinger, Young, Selby, & Shepardson, 2002). According to this categorisation, creativity involves critical thinking. Guilford supports that problem solving is the same phenomenon as creative thinking and through the process of creative production he specifies that the new solutions should be checked critically (Guilford, 1967). In order something to be perceived as creative it should have two main characteristics: to be original and useful (Rungo & Jaeger, 2012). Creativity characteristics vary within a person, no person can have all the creative characteristics and no person can behave according to those characteristics all the time (Treffinger, Young, Selby & Shepardson, 2002). On the other hand, the Critical Thinking Community consents to the definition of Schriven and Paul for critical thinking. According to this definition critical thinking ‘is the intellectually disciplined process of active and skillfully conceptualizing, applying, analyzing, synthesizing, and/or evaluating information gathered from, or generated by, observation, experience, reflection, reasoning, or communication’ (The Critical Thinking Community, 2013). This study accepts that both creativity and critical thinking are sub-categories of a broaden term which is the productive way of thinking (Newton, 2014).

The two traits measured were creativity and critical thinking and the two methods were written examinations and interviews. For the written examination of critical thinking there was a combination of the test of appraising observations (Norris & King, 1984) and the Cornell test for critical thinking level X (Ennis, Gardiner, Guzzetta, Morrow, Paulus & Ringel, 1964). For the interview of critical thinking the Ennis - Weir test was applied (1985). The letter of this test was read to the students by the researcher in order to overcome possible learning difficulties as the readability of the items are believed to play crucial role in exams (Hewitt & Homan, 2003). Specifically the test of appraising observations assesses the ability of the students to evaluate which statement is more believable. The Cornell evaluates exactly what Newton (2014) has defined as deductive reasoning, which is with specific conditions and “ifs” the students should come to a conclusion or as the same test supports ‘conditional reasoning’, while the Ennis-Weir demands evaluation of specific arguments. For the creativity in the written examination, a combination of a test of different uses for tools which resembles a test made by Guilford and also a pattern meanings test was used- both of the tests were taken by bibliography (Getzels & Jackson, 1962). The students were first asked to mention various uses of common objects and then various potential complete drawings. These tests do not have a single correct answer.The answers were evaluated according to their fluency (number of answers) and their uniqueness (originality). For the interview, a measurement tool of creativity was constructed, which asked students to narrate a fairy tale and evaluated their creativity according to the functions of the fairy tale (Propp, 1968), creative characteristics that can be in fairy tales (Rodari, 1996) and the presence of humour and violence in the story, which are two characteristics usually connected with the creativity (Getzels & Jackson, 1962). The tool mainly attempted to evaluate in verbal level the imagination and the innovation.

Method

The assessment took place in a secondary school in Greece with participants aged 13-15 years old. The validation of the measurement tools attempted to be done with the multi-trait multi-method matrix (Campbell & Fiske, 1959). The participants were solely 30 students, number which not allow us to extrapolate accurately to other groups. The students did not receive high grades in critical thinking tests and they were more efficient during the creativity tests. This is not an indicator that students are more creative than critical. There were more limitations in the critical thinking tests. The written tests for assessing critical thinking were multiple-choices tests which do not give the examinees the opportunity to justify their answers (Ennis,1993). Thus, the criterion used is not obvious, but there may be a justifiable criterion which was just different from the one which the researcher based on his attitudes, belies and context would have (Norris, 1989). The time given for answering the critical tests may be less than the demanded for students of a secondary school. Moreover, the students may have guessed the correct answer. The cultural context did not allow the students to interpret the argument of each paragraph correct as the behaviour in the roads was different in Greece. There was knowledge which students did not have in order to judge properly the arguments. The translation of the material and the adjustment according to the communicative language approach (Richards, 2005) did not help in the demonstration of the students’ critical thinking as they were notably attached to specific provided stimulus.

Expected Outcomes

The current study disputes reliability and validity of assessment tools which is not just attributed in the lack of justification in critical thinking tests and it supports another conclusion, which McPeck (1981) has already supported, that critical thinking cannot be faced as a subject independent skill which does not request prior knowledge. McPeck has supported that critical thinking is a skill which can be built in the knowledge of something, so we cannot talk about critical thinking in general, but first knowledge is demanded in order to build the ‘reflective scepticism’ on it. Similarly, creativity can be supported to be domain-specific (Zeng, Proctor & Salvendy, 2011) as the scores which students received in different uses test, pattern tests with drawings as stimuli and to verbal creativity differ with each other and they are not highly correlated. In addition, the multi-trait and multi-method matrix was not able to validate the use of these measurement tools. The rational explanation is the fact that each of these tools evaluated a different construct or even a different aspect of the same construct. This study does not imply that the creativity and critical thinking cannot be assessed, but it suggests a different series of assessment of creativity and critical thinking which is going to be subject-specific, while creativity and critical thinking are faced as high-order skills of specific constructs like the Bloom taxonomy would espouse (Krathwohl, 2002). It also recommends a new multi-trait multi-method analysis of these subject-specific tests in order to be examined if it is true that someone can be creative or critical in one domain, but not in another. In this case, when creativity and critical thinking are settled precisely to fit a singular subject, it means the inception of them being defined as subject-specific constructs which may lead in completely different perceptions of creativity and critical thinking.

References

Campbell, D.T. & Fiske, D.W. (1959). Convergent and Discriminant Validation by the Multitrait-multimethod matrix Ennis, R.H., Gardiner, W., Guzzetta, J., Morrow, R., Paulus, D. & Ringel, L. (1964). Cornell Critical Thinking Test Series. The Cornell Critical Reasoning Test. Form X. Illinois: University of Illinois at Urbana-Champaign. Ennis, R.H. & Weir, E. (1985). The Ennis-Weir Critical Thinking Test: Test, Manual, Criteria, Scoring Sheet. An instrument for teaching and testing. Pacific Grove: Midwest Publications Ennis, R.H. (1993). Critical Thinking Assessment. Theory into Practice, 32(3), 179-186 Getzels, J.W. & Jackson, P. W. (1962). Creativity and Intelligence: Explorations with Gifted Students. London and New York: John Wiley and Sons Inc. Guilford, J.P. (1967). The nature of Human Intelligence. Mc Graw-Hill Book Company Hewitt, M.A. & Homan, S.P. (2003). Readability level of standardized test items and student performance: The forgotten validity variable, Reading Research and Instruction, 43(2), 1-16 Krathwohl, D.R. (2002). A Revision of Bloom’s Taxonomy : an overview. Theory into Practice, 41(4), 212-218 McPeck, J.E. (1981). Critical Thinking and Education. Oxford: Martin Robertson & Company Ltd. McPeck, J. E. (1990). Critical Thinking and Subject Specificity: A Reply to Ennis. Educational Researcher, 19(4), 10-12 Newton, D.P.(2014). Thinking with Feeling: Fostering productive thought in the classroom. New York: Routledge Norris, S. P. & King, R. (1984). The design of a Critical Thinking Test on Appraising Observations. Studies in Critical Thinking. Research Report No1. Canada: Institute for Educational Research and Development, Memorial University of Newfoundland Norris, S.P. (1989). Can we test validity for critical thinking?. Educational Researcher, 18(9), 21-26 Propp, V. (1968). Morphology of the Folk Tale. Translation by Laurence Scott. The American Folklore Society and Indiana University Richards, J.C. (2005). Communicative Language Teaching Today. SEAMEO Regional Language Center Rodari, G. (1996). The Grammar of Fantasy: An Introduction to the Art of Inventing Stories. Translation and introduction by Jack Zippes. New York: Teachers & Writers Collaborative Rungo, M. A. & Jaeger, G.J. (2012). The Standard Definition of Creativity. Creativity Research Journal, 24(1), 92-96. Doi: 10.1080/10400419.2012.650092 The Critical Thinking Society (2013). Defining Critical Thinking. Check at : http://www.criticalthinking.org/pages/defining-critical-thinking/766 (Accessed: 28 January 2015). Treffinger, D.J., Young, G.C., Selby, E.C. & Shepardson, C. (2002). Assessing Creativity: A guide for educators. Sarasota, Florida: Center for Creative Learning Zeng, L., Proctor, R.W. & Salvendy, G. (2011). Can Traditional Divergent Thinking Tests Be Trusted in Measuring and Predicting Real-World Creativity?. Creativity Research Journal, 23(1), 24-37. Doi: 10.1080/10400419.2011.545713

Author Information

Ourania Maria Ventista (presenting / submitting)

Durham University, United Kingdom

Robert Coe (presenting)

Professor in the School of Education and Director of the Centre for Evaluation and Monitoring (CEM) in Durham University

Search the ECER Programme

Search for keywords and phrases in "Text Search"
Restrict in which part of the abstracts to search in "Where to search"
Search for authors and in the respective field.
For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.