In Turkey, as in many other countries (e.g. China, Greece, Iran, Japan, Russia, and Spain) university entrance exam is the sole criterion for student selection to higher education. The importance of the university entrance exam is increasing as millions of students sit in this exam and their results determine entry into universities or alternatives. In each year, high school graduates in Turkey take a stringent, centralized university entrance exam, seeking a place in one of the public or private universities. The competition is fierce and the exam content rigorous. In recent years, although the number of public and private universities increased, the seats at universities are still limited. In addition, willing to enter high prestige universities to gain employment and a higher status in society upon graduation still forces the competition.

University entrance exam determines the future of millions of young people and is the potent forces that influence the implemented curriculum in schools, particularly in high schools. Thus, the exam mathematics questions’ cognitive demands were examined by various researchers (e.g., Keleş & Karadeniz, 2015;), particularly using Bloom’s taxonomy. It is claimed that although the analyses of questions’ cognitive levels using Bloom’s taxonomy give an idea about their cognitive level, yet it is criticized by researchers (Tekkumru-Kisa, Stein, & Schunn, 2015) that “cognitive actions” as such application and evaluation do not constitute the cognitive demand and therefore could be low or high depending upon the nature of the situation. As pointed out by Tekkumru-Kisa, Stein, and Schunn, (2015), a task analysis guide (TAG) developed by Smith and Stein (1998) has proven to be useful both in research and practice setting. In the framework, there are four cognitive demand levels: 1) memorization, 2) procedures without connections, 3) procedures with connections, and 4) doing mathematics. While the levels, memorization and procedures without connections are categorized as “lower-level cognitive demands”, the other two levels, procedures with connections and doing mathematics are categorized as “higher-level cognitive demands”. The distinction made between the categories procedures with connections and procedures without connections is particularly useful. Procedures without connections refer broadly to “tasks that require students to perform a memorised procedure in a routine manner” (Stein et al., 2009, p. 1). Procedures with connections refer broadly to “tasks that demand engagement with concepts and that stimulate students to make powerful connections to meaning or relevant mathematical ideas” (ibid.). Although the framework was originally developed for instructional and curricular materials in mathematics and used in several studies in mathematics (e.g., Aysel, O’Shea, & Breen, 2011; Ubuz, Erbaş, Çetinkaya, & Özgeldi, 2010; Ubuz & Sarpkaya, 2014), in this study, it was adapted to be used in the analysis of mathematics and geometry questions in the university entrance examination. Cognitive demand is defined as “the kind and level of thinking required from students in order to successfully engage with and solve the task” (Stein, et al., 2000, p.11).

The main purpose of this study was to investigate the cognitive demand of mathematics and geometry questions in the Turkish university entrance examinations held between 2006 to 2017 (except 2014 and 2015). theThe findings of this study can be used for comparative studies of different countries assessment systems, and can provide insights into the improvement of mathematics education from an international perspective. The cognitive demands of the questions can shed new light on the TAG framework.

Mathematics and geometry questions of university entrance exams held between 2006 to 2017 (except 2014 and 2015) in Turkey were examined in this present study. The questions were developed and implemented by the Öğrenci Seçme ve Yerleştirme Merkezi (ÖSYM) (Measurement, Selection and Placement Center), and accessed from the web page of ÖSYM (ÖSYM, 2017a). ÖSYM was established by the Üniversitelerarası Kurul Başkanlığı (ÜAK) (Council of Intercollegiate) in 1974 (ÖSYM, 2017b). ÖSYM also develops and implements several nation-wide exams in addition to the university entrance exams. Between 2006 and 2009, students took one-stage exam, called Öğrenci Seçme Sınavı (ÖSS) (Student Selection Examination). The test was divided into two main parts: first part covered mostly 9th and 10th grade subjects (e.g., numbers, word problems) and rarely 11th grade subjects (mostly geometry subjects: circle, analytic geometry) and the second part included 11th and 12th grade subjects (e.g., complex numbers, functions) with some questions on 10th grade subjects (e.g. trigonometry). Even these questions were given in one stage, in the present study the first part questions were called as ÖSS-1 and the second part questions as ÖSS-2. Starting from 2010-2011 academic years, the exam was implemented in two-stage with different names, around two months between them: Yüksek Öğrenime Geçiş Sınavı (YGS) (Higher Education Exam) and Lisans Yerleştirme Sınavı (LYS) (Undergraduate Placement Exam). Students need to receive a score of 180 points in YGS (36% of the total points = 500) to be eligible to take the LYS. Generally speaking, the contents in ÖSS-1 and YGS, and ÖSS-2 and LYS are similar. For that reason, ÖSS-1 and YGS questions, and ÖSS-2 and LYS questions were considered together. Given that there are different names for exams, in this paper we prefer to use exam names as first and second stage university entrance exams. Multiple-choice questions with five options constituted the exams. Totally 360 mathematics and geometry questions (278 mathematics and 82 geometry questions) from the first stage exam (ÖSS1 and YGS) and 600 mathematics and geometry questions (386 mathematics and 214 geometry questions) from the second stage exam (ÖSS2 and LYS) were constituted the data sources of this study. Each question will be categorized based on the TAG framework by the researchers. Following this, the frequency of each demand across years and different subjects will be provided.

The cognitive demand levels of mathematics and geometry questions in national university exams in Turkey between 2006 and 2017 (except 2014 and 2015) will be categorized across the large range of years as well as across different mathematics and geometry subjects. This categorization is not only beneficial for Turkish University Entrance Examination System, but also international ones since many other countries have also their own standardized university entrance exams (e.g. Spain and Japan). Additionally, this study aims to improve Stein and Smith’s (1998) cognitive demand framework, by providing detailed descriptions of each cognitive demand. Although Stein and Smith (1998) provided several key characteristics for each category in order to categorize mathematical tasks (see Stein et al., 2000), we believe that by specifying definitions of each demand in the current framework, we can identify the related cognitive demand levels of different kind of tasks much easier by using action verbs (e.g. execute, formulate, investigate, create). Moreover, in this way, we will contribute to related literature in specifying different cognitive demand levels by using concrete examples.

Aysel A., O'Shea, A., & Breen, S. (2011). A classification of questions from Irish and Turkish high-stakes examinations. In Proceedings of the British Society for Research into Learning Mathematics. BSRLM, London, pp. 13-18. Keleş, T., & Karadeniz, M. H. (2015). An analysis of mathematics and geometry questions in OSS, YGS and LYS according to the revised bloom taxonomy between 2006-2012 years. Turkish Journal of Computer and Mathematics Education (TURCOMAT), 6(3), 532-552. ÖSYM (Ölçme, Seçme ve Yerleştirme Merkezi) (2017a). ÖSYS: Öğrenci Seçme ve Yerleştirme Sistemi Sınav Soru ve Cevapları. Retrieved from http://www.osym.gov.tr/TR,12910/2017.html. ÖSYM (Ölçme, Seçme ve Yerleştirme Merkezi) (2017b). ÖSYM Tarihçe. Retrieved from http://www.osym.gov.tr/TR,8789/hakkinda.html. Stein, M. K., & Smith, M. S. (1998). Mathematical tasks as a framework for reflection: From research to practice. Mathematics Teaching in the Middle School, 3(4), 268-275. Stein, M. K., Smith, M. S., Henningsen, M., & Silver, E. (2000). Exploring cognitively challenging mathematical tasks: A casebook for teacher development. New York: Teachers College Press. Stein, M. K., Smith, M. S., Henningsen, M. A., & Silver, E. A. (2009). Implementing standards based mathematics instruction, A casebook for professional development (2nd ed.). Reston, VA: National Council of Teachers of Mathematics. Tekkumru-Kisa, M., Stein, M. K., & Schunn, C. (2015). A framework for analyzing cognitive demand and content‐practices integration: Task analysis guide in science. Journal of Research in Science Teaching, 52(5), 659-685. Ubuz, B., Erbaş, A. K., Çetinkaya, B., & Özgeldi, M. (2010). Exploring the quality of the mathematical tasks in the new Turkish elementary school mathematics curriculum guidebook: The case of algebra. ZDM, 42(5), 483-491. Ubuz, B., & Sarpkaya, G. (2014). The investigation of algebraic tasks in sixth grades in terms of cognitive demands: Mathematics textbook and classroom implementations. Elementary Education Online, 13(2), 594-606.