As educational researchers professionally responsible for the search of truth, politicians, on the other side, should feel obligated in making scientific results effective for public goods.  Therefore, educational researchers should elucidate that political decisions without a scientific background will have been interrogated and, eventually, refuted. In many instances, the commitment of scientific endeavor for the educational progress is turned down by politicians. On the other hand, there are examples that decision makers have a tendency to use and cite scientific results to commercialize their popularity in the public eye even without comprehending the meaning of the study itself. Nevertheless, the real dilemma on today's educational research, especially in Turkey, is about research significance. That is to say, whether conducting a research that bases its roots on overnight political declarations--about ameliorating educational system-- or conducting a research that becomes a base for accurate political decisions. 

Based on aforementioned, similar to many studies that take their departure point from overnight decisions of politicians, this study concentrates one of the contemporary debates about the transition to Kazakh system in measuring student success through nation-wide exams at middle-school level. Recently, Minister of National Education, in Turkey, expressed an intention, by including open-ended questions, to increase higher order thinking skills and, thus, made initiations in converting examination style to more thinking oriented one in evaluating student success (Berberoglu, 2009; Demirtasli, 2010; TEDMEM, 2013; MoNE, 2017). Not only in Turkey, but many other countries, have similar concerns and in search for developing effective knowledge construction. The obvious truth in many educational settings is that individuals' interaction with the question types seems as the only effective way whether knowledge construction happened. However, there should be a congruity between utilizing scientific truth and governing political power. Throughout the history examples exist that depicture how scientists and, thus, educational research are oppressed by political regimes. More specifically, this study aims to underline some scientific backgrounds before any transitional process occurs. It is conducted to resolve the conflict in Turkey about the fact that whether Kazakh evaluation system by employing open-ended question format must definitely be used in large-scale assessments such as Transition from Primary to Secondary Education Exam (MoNE, 2013) by disregarding multiple-choice exams (Berberoglu & Is-Guzel, 2013).  For this purpose, the main aim of this study is to explore the comparative effects of open-ended and multiple-choice exams with regard to metacognitive and affective dimensions according to new large-scale examination system in Turkey. Specifically, pertinent to this study is to show differential effects of question formats (Bridgeman, 1992) especially in mathematics assessment to government officials before any actual country-wide implementation. First of all, the study will reveal what people think about usability and feasibility of this new assessment movement and how the replacement of MC with OE is perceived from different perspectives (Birenbaum & Feldman, 2006). Also, the study is aimed to investigate the differential effect of MC and OE questions on students' metacognitive and affective characteristics in terms of worry, effort, self-checking, and cognitive strategy (Efklides, 2011; O’Neil & Brown, 1998). Can open-ended questions be a solution to current problems in terms of the transition from middle to high school in Turkish educational system? is the main research question and, in this regard, this study discusses how differences between MC and OE questions in terms of their strength and weaknesses perceived, classifies metacognition and cognitive strategy, searches affect by its effort and worry constructs in solving open-ended questions, discusses the appropriateness of MC and OE exam types in large-scale assessments from the viewpoint of middle school students, their teachers, and academicians, and, finally, provides suggestions to public officials. 

In this study, metacognitive and affective dimension of MC and OE questions were examined through exploring the experiences and views of 8th-grade students, their teachers, and academicians. Instead of unveiling participants' perceptions about the question formats by using quantitative surveys or scales, researchers used the phenomenological approach (Patton, 1990) which requires an active participation. This phenomenological inquiry involved 32 participants for an in-depth understanding of metacognitive and affective dimensions. To gather proper data for both exam types, parallel question forms were developed and used for inductive and holistic understanding of human experience in a context specific settings. For instance, instead of asking students to solve a specific question types, researchers posed questions about the question structures of MC and OE, required metacognitive skills to understand and solve the problem, evidences that help to comprehend what the question really asks, how to reword the question, rewording multiple-choice question to open-ended format, and so forth. By conducting a cognitive interview (Willis, 1999) respondents revealed information essential to explicate how question types and their solutions may differ in terms of metacognitive and affective dimensions of exam types. In addition, verbalization of thoughts and feelings were inquired and observed while examining the knowledge necessary to solve a specific type of question and "think aloud" and "verbal probing" approach (Ericsson & Simon, 1990) were employed to understand how participants differentiate between two types of exams. This approach, gathering qualitative (narrative) data in the holistic inductive paradigm, provided in-depth and detailed information, evidence for describing the exam phenomena from participants' views through direct quotations, observations, interviews, and document analysis. Data gathered during the process were analyzed through content analyses method. Codes, categories, and themes were developed based on information gathered from interviews, observations, and document analysis (Patton, 1990). To do this, researchers followed the methodological process by interviewing coherently, using an appropriate number of participants, maximizing dynamic relation between data saturation, and analyzing data without overlooking the theoretical framework. In this study, the theoretical framework was restricted to discuss differentiated effects of MC and OE consistent with two sub-dimensions of metacognition and affect from the experiences of middle school teachers, 8th-grade students and academicians to satisfy comprehensibility of the research.

This study has two-dimensional effect. First one aimed to explore the comparative effects of open-ended and multiple-choice exams with regard to metacognitive and affective dimensions. This indicated that instead of using multiple-choice exams to measure student success and use this result for nation-wide exam result for transition purposes from middle school to high schools, open-ended questions would be preferred since they intend to measure student achievement better. However, results of this study indicated that both examination types have positive and negative sides with regard to cognitive and affective dimensions. Having said that, this phenomenological study underlines the fact that both question types trigger different sorts of cognitive strategy. For instance, some students prefer multiple-choice more because self-checking through item stem and alternatives is easier while some prefer open-ended since it provides chances to look at the question from a broader perspectives. In addition, some students and teachers indicated that multiple-choice provides content validity benefits while open-ended is limited at this part. Moreover, findings showed that effort spent in solving questions differs among high, middle, and low achievers. Interestingly, there is no consensus among the level of achievers in relation to spending an effort on different types of questions. For instance, some high achievers indicated that open-ended requires more effort while others mentioned as less effortful. In addition, opinions differentiated in relation to anxiety and worry. While both exam types create less anxiety on high achievers, middle and low achievers demonstrate differences in anxiety levels. Based on the results of this study, changing the question type is not a spontaneous heal in improving student placement via test results (Heck & Stout, 1998; Johnson, Sieveking, & Clanton, 1974; Lawrenz, Huffman, & Welch, 2000; O’Neil & Abedi, 1992; Ozuru, Briner, Kurby, & McNamara, 2013).

Berberoglu, G. (2009). CİTO Türkiye öğrenci izleme sistemi (ÖİS) öğrenci sosyal gelişim programı’na (ÖSGP) ilişkin ön bulgular. [CITO Turkey student follow- up system (OIS) pre-findings about student social development program.] CITO Education: Theory and Practice Journal, November-December Issue, 32-42. Berberoglu, G., & Is-Guzel, C. (July-September, 2013). Eğitim sistemimizdeki ölçme ve değerlendirme nasıl olmalıdır? [How should educational measurement and evaluation practices be in an educational system?] CITO Education: Theory and Practice Journal, Issue, 21, 9-16. Birenbaum, M., & Feldman, R. A. (2006). Relationship between learning patterns and attitudes towards two assessment formats. Educational Research, 40(1), 90-98. Bridgeman, B. (1992). A comparison of quantitative questions in open-ended and multiple-choice formats. Journal of Educational Measurement, 29(3), 253-271. Demirtasli, N. (2010). Açık uçlu soru formatı ve öğrenci izleme sistemi (ÖİS) akademik gelişimi izleme ve değerlendirme (AGİD) modülündeki kullanımı. [Open-ended question format and student monitoring system (OIS) academic development monitoring and evaluation] CITO Education: Theory and Practice Journal, April-June Issue, 21-30. Efklides, A. (2011) Interactions of metacognition with motivation and affect in self- regulated learning: The MASRL model. Educational Psychologist, 46(1), 6-25. Lawrenz, F., Huffman, D., & Welch, W. (2000). Policy considerations based on a cost analysis of alternative test formats in large-scale science assessments. Journal of Research in Science Teaching, 37(6), 615-626. Ministry of National Education [MoNE/MEB] (2013) Temel Eğitimden Ortaöğretime Geçiş. [Transition from Primary to Secondary Education.] Retrieved on June 25, 2014, from http://oges.meb.gov.tr/docs2104/sunum.pdf Ministry of National Education [MoNE/MEB] (2017) Açık Uçlu Sorular Önemli Bir Dönüm Noktası Olacaktır [Open-ended Questions will be an Important Milestone] Retrieved on February 08, 2017, from http://www.meb.gov.tr/bakan-yilmaz-anadolu-ajansi-editor-masasi8217na-konuk-oldu/haber/12887/tr O'Neil, H. F., Jr., & Abedi, J. (1992). Japanese children's trait and state worry and emotionality in a high-stakes testing environment. Anxiety, Stress, and Coping, 5(1), 253-267. O’Neil, H. F., & Brown, R. S. (1998). Differential effects of question formats in math assessment on metacognition and affect. Applied Measurement in Education, 11(4), 331-351. Ozuru, Y., Briner, S., Kurby, C. A., & McNamara, D. S. (2013). Comparing comprehension measured by multiple-choice and open-ended questions. Canadian Journal of Experimental Psychology, 67(3), 215-227. Patton, M. Q. (1990). Qualitative evaluation and research methods. (2nd ed.). USA: Sage Publications. Willis, G. B. (1999). Cognitive interviewing: A “How To” guide. Meeting of the American Statistical Association.