Validation of the Verbal Test of Scientific Imagination

Author(s):

Ying-Yao Cheng(presenting / submitting)Chia-Chi Wang Hsiao-Chi Ho Chih-Ling Cheng Hung-Lin Chiu

Conference:

ECER 2014

Network:

09. Assessment, Evaluation, Testing and Measurement

Format:

Poster

Session Information

09 SES 05.5 PS, General Poster Session

General Poster Session

Time:

2014-09-03

12:30-14:00

Room:

Poster Area E (in front of B001-B003)

Chair:

Contribution

1. Introduction

Imagination and creativity are the curial element to dominate the present economy and culture (McCormack, 2010). Today’s scientific education is the best opportunity for emphasizing imagination and innovation. However, there is less literatures to discuss the importance of imagination in science education. In order to understand imagination in science education, Ho, Wang, and Cheng (2013) proposed that the scientific imagination process was constructed based on the professional knowledge and rich experiences of award-winning teachers (Ho et al., 2013).They divided the process into three stages: Initiation, Dynamic Adjustment, and Virtual Implementation. Four different key components operate during each of the three stages: Brainstorming, Association, Transformation/Elaboration, and Conceptualization/Organization/Formation. Based on this process (Ho et al., 2013), this study aims to develop an verbal test to measure pupils’ scientific imagination for 5^th and 6^th graders.

In recent years, Rasch techniques have been widely applied to test quality of questionnaires in various fields such as human science, education, medical science and management (Wang, 2004). Rasch techniques possess objective and interval properties and thus can improve limitations of classical test theory and benefit to test quality of assessment through multiple sources of evidence. Therefore, we utilize the Rasch techniques to conduct the validation of SIT-Verbal in the study.

Method

2.Method 2.1 Participants Two samples of grade 5 and grade 6 elementary school students participated in the validation of the SIT-Verbal in southern Taiwan, one for item revision and the other for validation. Sample 1 was 558 participants (287 boys, 268 girls, and three unidentified) and sample 2 was 183 participants (92 boys, 91 girls). 2.2 Instruments The Scientific Imagination test-Verbal (SIT-Verbal) was used in this study. The SIT-Verbal was a situation test which designed to measure pupils’ scientific imagination for 5th and 6th graders. This test was covered four components, Brainstorming, Association, Transformation/Elaboration, and Conceptualization/Organization/Formation, contained six situation questions. Taking the first item as an example, “what problems might the situation bring about?” Each item has four point response categories from 0 (lower) to 4 (upper). 2.3 Analysis The Rasch partial credit model (PCM; Masters, 1982) was employed to fit to the data. The parameters in the PCM were estimated using the computer program ConQuest (Wu, Adams, & Wilson, 2007). Data analysis comprised two phases—one for item revision and the other for validation. During the first phase, the quality of individual items is evaluated. When the data fit the model’s expectation, the infit (weighted) and outfit (unweighted) mean square error (MNSQ; range: 0.7~1.3) has an expected value of unity (Wright & Linacre, 1994). In the validation phase, we employed the PCM to examine the fitting quality of each item and the unidimensional construct of scientific imagination. Next, with respect to the generalizability aspect of validity, we conducted differential item functioning (DIF; Holland & Wainer, 1993) analyses among genders.

Expected Outcomes

3. Results In sample 1, unidimensional model-data fit analyses were conducted respectively for the SIT-Verbal using the PCM. All items had good model-data fit, except the item six. Thus, we deleted it in this test. In validating phrase, all the items had good model-data fit. Related to the generalizability aspect of validity, the results showed that there are no statistically significant differences between genders in scientific imagination. The reliability of the scientific imagination was .85. Furthermore, related to substantive evidence, the item difficulty of transformation/elaboration is highest (1.63 logits), but the item difficulty of brainstorming is lowest (-1.40 logits). 4.Discussion and conclusions The results showed that the SIT-Verbal had a reasonably good model-data fit. The person separation reliability of the measure was .85. No significant DIF was found across gender. The SIT-Verbal eventually consisted of five items. It was concluded that the SIT-Verbal is a reliable and valuable open ended tool for assessing individual’s scientific imagination, future research should address how the SIT-Verbal can be applied to the development of training courses to foster students’ scientific imagination.

References

1. Ho, H. C., Wang, C. C., & Cheng, Y. Y. (2013). Analysis of the Scientific Imagination Process. Thinking Skills and Creativity, 10, 68-78. 2. Holland, P. W., and Wainer, H. (Eds.) (1993). Differential item functioning. Hillsdale, NJ: Lawrence Erlbaum. 3. Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47(2), 149-174. 4. McCormack, A. J. (2010). Imagine and invent: Create a great future. Journal of College Science Teaching, 22, 14-15. 5. Wang, W.-C. (2004). Rasch measurement theory and application in education and psychology. Journal of Education and Psychology, 27, 637-694. 6. Wright, B. D., & Linacre, J. M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8, 370. 7. Wu, M. L., Adams, R. J., & Wilson, M. R. (2007). ConQuest [Computer software and manual]. Camberwell, Victoria, Australia: Australian Council for Educational Research.

Author Information

Ying-Yao Cheng (presenting / submitting)

National Sun Yat-sen University

Institute of Education

Kaohsiung

Chia-Chi Wang

National Sun Yat-sen University, Taiwan

Hsiao-Chi Ho

National Sun Yat-sen University, Taiwan

Chih-Ling Cheng

National Sun Yat-sen University

Faculty of Education

Kaohsiung

Hung-Lin Chiu

National Kaohsiung Normal University, Taiwan

Search the ECER Programme

Search for keywords and phrases in "Text Search"
Restrict in which part of the abstracts to search in "Where to search"
Search for authors and in the respective field.
For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.