Comparative Analysis of Essay Assessment By Two Methods: Criteria-Based Assessment And Comparable Judgement at Kazakhstan Schools

Author(s):

Tatyana Shikhareva(presenting / submitting)Gulnur Izbassova(presenting)

Conference:

ECER 2017

Network:

09. Assessment, Evaluation, Testing and Measurement

Format:

Paper

Session Information

09 SES 02 B, Issues in Assessing Reading and Language Competencies in Mother Tongue, Second and Foreign Language

Paper Session

Time:

2017-08-22

15:15-16:45

Room:

W5.18

Chair:

Sarah Howie

Contribution

In the modern educational space there have been great changes in the evaluation system due to the introduction of new approaches in education. A new evaluation system has been implemented in The Nazarbayev Intellectual school (NIS) system, including the criteria-based assessment. Implementation of the new evaluation system in the school practice requires analysis, and improvement of evaluation in order to improve the quality of teaching (Kavetsky, Ryzhkovskaya, 2010) [1].

For the system of secondary education of the Republic of Kazakhstan criteria-based assessment is implemented in 2011, for a total of 6 years in the pilot schools, so this kind of evaluation is new for country. This raises a lot of questions and problems

Criteria-based approach (CBA) to the evaluation of students’ works become more and more popular in modern school, probably because such approach helps to exercise, particularly, feedback that enables all participants of the process (teachers and pupils, and parents). (CBA) involves a number of basic settings for the teacher:

1 Only the work of the student, not his personality can be evaluated using a mark;

2. Work the student is not directly comparable with the work of other students, but with the standard ( the way of perfect work);

3. The standard is informed to students in advance;

4. A clear marking algorithm is developed by which the student can determine his level of achievement and determine his mark by himself;

5. Teachers can evaluate only what is taught, so the evaluation criterion - a concrete expression of the educational objectives;

6. General learning objectives in the subject are the criteria for assessment of student achievement in the subject and allow to generalize the results, identified from student's separated works [2]

Why did we choose this research topic?

According to the results of academic progress on World History and English language among seventh grade with Kazakh language training a problem has been revealed - students can not write an essay.

The reasons: the language barrier, small vocabulary, did not read the task attentively, do not comply with the format of the essay (introduction, main part, conclusion), do not reveal the topic, do not know how to argue, give examples and facts, can not draw conclusions.

To solve this problem, teachers of World History and the English language have decided to conduct a study mong 7th grades with employment of Comparable Judgement approach.

Actuality of the topic: teachers of language subjects (Russian, Kazakh, literature and English), geography, history of Kazakhstan are faced such problems.
Currently, new requirements for the competence of pupils are presented, such as functional literacy, so this skill is very important as the ability to write an essay .

The purpose of the research: objective assessment of student’s work using the approach, to identify whether evaluation criteria are drawn up correctly and how to draw p evaluation criteria

Tasks:

To teach students to write an essay on a specific topic in Russian and English languages

To improve the quality of knowledge and skills

Increase vocabulary

To teach to follow the logic order of writing essays, taking into account characteristics of the subject being studied

To teach to follow the rules of grammar, spelling and punctuation

Teach to reveal the theme of essays deeply.

Hypothesis: Using different methods of estimation (criteria-based and comparative) teachers will be able to formulate the most effective criteria for assessing the essay that will help students to cope with this task.

Methods: The evaluation criteria, comparative evaluation, analysis, synthesis, and interviews.

The novelty of the research: There are no similar studies have been conducted in Russian language

Method

Having identified the problem, we drew up an action plan. Action plan: Phase 1: to choose a topic essay, class Phase 2: to check the essays using criteria-based assessment We check the essay by Comparable Judgement (CJ) (reviews interviewers about essay, which, in their opinion, are very good work, poor work, an average work and why) Phase 3 We analyze the results of the evaluation using two methods: the CBA and CJ We compare the results and identify common problems and solutions to these problems; We draw conclusions. Phase 4: Reflection. What are the problems we face in CBA, when and where to apply these methods, what remains unclear, in which direction to continue the research on this topic, difficulty and risk, the prospect of such a method. Subject: English and World History, class 7, 10 students. Justification: a comparison method is suitable only to the humanities. We have chosen a topic of research, research methods, research objective and predictable results, receptions, class and students We identified essay topic in Russian and English (duplication) on World History. Justification: why we chose the theme of global history, and not in English. Because essays topics in the English language program are free, and on World History essay topics associated with a particular historical event or process, or with a particular historical figure. 10 pupils have written an essay on "The Causes of quick wins of Mongol invaders" in Russian and English languages. Teachers have checked the works using criteria-based assessment and other teachers were asked to read an essay and give a comment (to choose, in their opinion, the best, average and weak essay) and explain their choice. How do the criteria for assessment of World History and the English language differ? On the World History the author of the essay should know and intelligently use theoretical (historical) concepts and terminology and should be able to analyze the historical events, to explain the causes and consequences of historical facts, events, and processes. Stella Cottrell [2] believes that for writing critical analytical writing important letters to the author evaluated the strengths and weaknesses, compare the information to make informed judgments, reasoned, in accordance with the evidence. In the English language to test writing skill there are standard evaluation criteria including linguistic aspects such as grammar and vocabulary, structure, the presence of logical order and content

Expected Outcomes

Drawbacks of CJ 1. Essay checker does not see a plan of essay (what should be the content of the essay), that’s why there are differences of assessment 2. Essay checker does not know the required length of essays All these factors entail not entirely objective assessment of essays. Benefits of CJ 1. There is no human factor, because essay checker does not know the author of the essay, does not know his level of English, which creates transparency essay assessment 2. In contrast with subjects such as the History of Kazakhstan, World History, Geography, etc. in language subjects, there are standard and mandatory criteria that every teacher is guided and apply to any kind of essay, regardless of the topic. They are checking of grammar, vocabulary, structure and logical order. Conclusion: Studying the estimation of World History and English language the same trend can be traced- Both CBA and CJ were used to evaluate the same works of the same students. But there was a difference in determining the best, medium, and low performance. The cause is incorrect definition of the criterion of "the weakness of the conquered countries." The quality of the writing essay also depends on the language competence of students. For students who wrote essays in Russian language (World History) Russian is a second language and English is a third language. Not only knowledge of the topic and understanding of the criteria and strict adherence to the criteria affect the quality of essay writing, and, accordingly, the assessment Thus, improving the assessment criteria that give opportunity to the student to know precisely his tasks and what actions he should perform to complete the task.

References

1. IT Kavetsky, TL Ryzhkovskaya, IA Koverzneva, VG Ignatovich, NA Loban, SV Starovoitova. Fundamentals of psychology and pedagogy - Minsk. Publishing house MIM 2010 2. Stella Cottrell (2008) Guidance on study skills, 286. 3. http://nao.kz/content/view/7/260, National Academy of Education named Y. Altynmarina, 2017 4. Laming, DRJ (2004) Human judgment: the eye of the beholder. London, Thomson. 5. Thurstone, LL (1927a). Psychophysical analysis. American Journal of Psychology, 38, 368-389. Chapter 2 Thurstone, LL (1959). Measurement of values. University of Chicago Press, Chicago, Illinois. 6. Thurstone, LL (1927b). The method of paired comparisons for social values. Journal of Abnormal and Social Psychology, 21, 384-400. Chapter 7 Thurstone, LL (1959). Measurement values. University of Chicago Press, Chicago, IL 7. Bramley, T (2007) Paired comparison methods. In Newton, P, Baird, J, Patrick, H, Goldstein, H, P, Timms and Wood, (eds) Control methods of expertise standard comparability. London standards, CSA. 8. B Pollitt, A (2012) The Adaptive comparative judgment method. Evaluation in Education: Principles, Policy and Practice. 19: 3, 1-20. DOI: 10,1080 / 0969594X.2012.665354 9. www.digitalassess.com 10. Kimbell R, A and Pollitt A (2008) assessment Coursework high-stakes exams: authenticity, creativity, reliability of the third international conference Rasch measurement. Perth: Western Australia in January

Author Information

Tatyana Shikhareva (presenting / submitting)

Nazarbayev Intellectual school of Aktau city

History and English

Aktau

Gulnur Izbassova (presenting)

Nazarbayev Intellectual School of chemistry and biology in Aktau

English language