The Development Of A Curriculum-Based C-Test For Young EFL Learners
Author(s):
Raphaela Porsch (presenting / submitting) Eva Wilden
Conference:
ECER 2015
Format:
Paper

Session Information

09 SES 10 A, Assessments in Second-, Bi- and Multi-Language Settings

Paper Session

Time:
2015-09-10
15:30-17:00
Room:
326. [Main]
Chair:
Heidi Harju-Luukkainen

Contribution

Over the past 15 years, there have been a number of significant changes within the German educational system as illustrated in the following examples: the practice of individually fostering each school pupil is now legally binding. The evaluation of learner achievements through teachers in the classroom is of particular importance in order to gain knowledge about individual development as a basis for planning lessons that take individual differences of students into account. One vital point of time for the assessment of students’ competencies is when teachers start teaching a new class as they need reliable information about their students’ prior knowledge. In Germany the majority of students are making the transition from primary to secondary school grade 4 (age app. 10). Thus, teachers at the beginning of grade 5 need a variety of practical methods and instruments to evaluate the new students.

The project to be presented explores the possibility of diagnosing the English as a Foreign Language (EFL) competence of young learners at the end of primary school with the help of a newly developed C-test. This C-test is designed to evaluate students’ EFL competence at the beginning of secondary school. C-tests aim at measuring general language ability (e.g., Asano, 2014) and have proven to be an effective instrument for assessing learners of different languages and in various age groups – however they have not yet been adapted for very young EFL learners. The C-test is based on the „reduced redundancy principle“ (cf. Babaii & Ansary, 2001). In contrast to Cloze-tests, in C-tests only parts of words are deleted, not whole words. Ideally, C-tests consist of four to five short texts that are content neutral, suitable for the target group, non-fictitious, without dialogues and authentic. Each text consists of 20 to 25 gaps with 80 to 100 words whereby every second or third word is mutilated.

Along with the development of C-tests, several studies have researched the use of test-taking strategies while taking a C-test (e.g., Grotjahn & Stemmer, 2002; Salehi & Sanjareh, 2013). Cohen (2011, p. 317) summarizes the findings of research on C-test-taking strategies as follows: “It has been found that response strategies predominantly involve micro-level processing. Since half the word is given, students who do not understand the macro-context have been observed to mobilize their vocabulary skills in order to fill in the appropriate discourse connector without indulging in higher level processing.” This seems to be true for language learners at a beginning and an intermediate level of proficiency.

In developing the research design the following principles need to be adhered to: (1) following the requirements of C-test criteria as sketched out above, especially following the redundancy principle; (2) take the prerequisites of the young learners into account, especially by referring to the curriculum with regard to familiar topics, vocabulary, grammar, reading and writing competencies; (3) developing an instrument that allows reliable and valid testing of EFL proficiency, and (4) including the children’s perspectives in the test development.

Based on these criteria, our research questions are as follows:

  1. Psychometric quality of the test: Does the C-test show appropriate levels of item difficulty and item discrimination as well as a satisfactory reliability?
  2. Validity of the C-test: Is there at least a medium correlation between the C-test scores and the grades in English being a first indicator of the test’s construct validity?
  3. Test takers’ usage of strategies: Which test-taking strategies do fourth-graders apply when solving the C-test based on written reports?

Method

Participants: In 2014 data was collected at four primary schools from 201 fourth-graders with 3.5 years of EFL education. The students’ mean age was 9.93 years (SD = .53). 49% of the students were male, 51% were female. 55.1% of the students stated that German was their only language spoken at home. The distribution of the grades in English (EFL) at the end of primary school is as follows: 1 (“very good”): 13.7%, 2 (“good”): 50.5%, 3 (“fully satisfactory”): 28.9% and 4 (“still satisfactory”): 6.8%; grade 5 and 6 was not given. Instrumentation: The curriculum for English from the German state of North-Rhine-Westphalia (MSW, 2008) states that children at the end of primary school reach CEF level A1. Required EFL writing skills are limited to copying words and individual sentences. Since C-tests should consist of authentic texts, we choose texts from standard textbooks, children’s books, story cards, and children’s stories published in didactic journals. Next to the text type, we needed to find texts with familiar vocabulary. The curriculum states five different “fields of experience” with sub-topics (ibid., p. 76); we aimed at covering at least three of the fields with our texts. Finally, in the study four texts with twenty gaps each were used. A further issue with C-tests is the question which scoring methods should be applied. There are two possible methods; both were applied in our study: (A) dichotomous coding that is widely used for analyzing C-tests (cf. Eckes & Grotjahn, 2006) or (B) a differentiated evaluation leading to polytomous items. Method B takes the idea of Communicative Language Teaching into account, an approach that “emphasizes learning a language first and foremost for the purpose of communicating with others” (Duff, 2014, p. 15). The four C-test texts and the scoring methods were used for the first time in a pilot study with 144 fourth-graders in 2013. The study follows a mixed method approach. Immediately after testing, the students had to report on the employment of strategies. From each text a short passage was presented comprising three sentences along with the question “How did you find the solution?” and space to be answered by the test-takers. Altogether from 102 children answers about the employment of strategies could be used for qualitative content analysis (cf. Schreier, 2012).

Expected Outcomes

All texts were moderately difficult and within a range from .38 to .47 which indicates that all items – all subtests within the C-test – could be used for further testing. With rit ≥ .71 for all items, it means that the texts are well discriminating (see Fisseni, 2003, p. 80). The reliability of the test is sufficient as well (α ≥ .80; ibid.). With regard to research question 2 we find significant correlations with a range of -.40 to -.50 which can be interpreted as a first indicator of construct validity and that the developed C-test measures English proficiency. To sum up, as the first analyses show, reliable and valid assessment is possible by using this C-test for young EFL learners. Sixteen different strategies could be identified when solving the C-test: 13 reading strategies and 3 test-management strategies. The majority of the reading strategies were micro-level strategies. Predominantly researchers have used the think-aloud technique to collect information about test-takers’ usage of strategies. However, there are some limitations in collecting data from large groups. Since more than half of the children in this study responded meaningfully to the open-answer questions, we argue that for this purpose – questioning more than two hundred children – the coding of written responses provides an alternative. In addition to the presentation of further results (e.g., proficiency scores by subgroups), the benefits and challenges in the development of a C-test for this particular age group along with its practicability for assessment purposes in the German and European context, and using the written response method to explore the strategy usage of young children will be discussed.

References

Asano, Y. (2014). C-Tests und ‚allgemeine Sprachkompetenz‘: Theoretische Überlegungen und empirische Analysen. In R. Grotjahn (Ed.), Der C-Test: Aktuelle Tendenzen/The C-Test: Current Trends (pp. 39-52). Frankfurt/Main: Peter Lang. Babaii, E. & Ansary, H. (2001). The C-test: a valid operationalization of reduced redundancy principle? System, 29, 209-219. Cohen A. D. (2011). Strategies in learning and using a second language. Harlow, England: Longman Applied Linguistics/Pearson Education. Duff, P. A. (2014). Communicative language teaching. In M. Celce-Murcia, D. M. Brinton & M. A. Snow (Eds.), Teaching English as a Second or Foreign Language (pp. 15-31). Boston: National Geographic Learning. Eckes, T. & Grotjahn, R. (2006). A closer look at the construct validity of C-tests. Language Testing, 23 (3), 290-325. Fisseni, H. J. (2003). Lehrbuch der psychologischen Diagnostik: Mit Hinweisen zur Intervention. Goettingen: Hogrefe. Grotjahn, R. & Stemmer, B. (2002). C-Tests and language processing. In J. A. Coleman, R. Grotjahn & U. Raatz (Eds.), University Language Testing and the C-Test (pp. 115-129). Bochum: AKS. MSW (Ministerium für Schule und Wissenschaft Nordrhein-Westfalen) (2008). Richtlinien und Lehrpläne für die Grundschule in Nordrhein-Westfalen. Düsseldorf: MSW NRW. Salehi, M. & Sanjareh, H. (2013). On the comparability of C-test and Cloze: A verbal protocol approach. English for Specific Purposes World, 14 (39). Online: http://www.esp-world.info/Articles_39/Abstract_39_Salehi_Sanjareh.htm Schreier, M. (2012). Qualitative Content Analysis in Practice. London: SAGE.

Author Information

Raphaela Porsch (presenting / submitting)
University of Muenster, Germany
University of Vechta, Germany

Update Modus of this Database

The current conference programme can be browsed in the conference management system (conftool) and, closer to the conference, in the conference app.
This database will be updated with the conference data after ECER. 

Search the ECER Programme

  • Search for keywords and phrases in "Text Search"
  • Restrict in which part of the abstracts to search in "Where to search"
  • Search for authors and in the respective field.
  • For planning your conference attendance, please use the conference app, which will be issued some weeks before the conference and the conference agenda provided in conftool.
  • If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.