Using mixed methods for assessing the qualitative and quantitative aspects of spelling following a language based metacognitive intervention
Author(s):
Conference:
ECER 2010
Format:
Paper

Session Information

09 SES 08 C, Issues in Language and Literacy Assessment

Paper Session

Time:
2010-08-26
17:15-18:45
Room:
P617, Porthania
Chair:
Ruth Zuzovsky

Contribution

This paper reports on the qualitative and quantitative assessment of eight pupils with spelling difficulties. Therapy interventions used a language based metacognitive approach to literacy development. In this paper the emphasis is on spelling but other aspects of literacy development, such as reading, were also assessed. Instructional practices focused on developing the pupil’s active participation within the learning process. For each pupil this was promoted through the development of metacognitive knowledge and experience, (Flavell 1979), at person, task and strategy level. Ultimately the therapy aim was to develop pupils’ self efficacy beliefs and empower them to take responsibility for their learning.

Spelling is a neglected area of research in comparison to its counterpart reading, (Leask, 2007; Apel et al., 2001). This is evident within the area of assessment and evaluation of change following an intervention programme. The key objective is to develop pedagogic approaches which are likely to extend and develop a pupils’ spelling skills. The research questions posed were how can relating quantitative and qualitative methods of assessment and evaluation inform pupil progress and how can this inform mainstream pedagogic practices?

Spelling is impacted on and influenced by linguistic factors, such as phonology, (Apel et al 2001), but other language dimensions such as semantics and morphology, syntax and pragmatics are also involved, (Vellutino et al., 2004). It is therefore crucial that these linguistic dimensions are accounted for when developing assessment techniques and approaches to analysing spelling. Taking this quantitative and qualitative approach, increasingly referred to as mixed methods, (Tashakkori, & Creswell 2007b), can only aid in the quality of assessments and interpretation of presented spelling data.

The challenges and opportunities of taking a mixed methods approach is well documented with the literature (e.g. Delinger, & Leech 2007; Greene 2006; Johnson, Onwuegbuzie & Turner, 2004; Yin, 2003).  A mixed methods approach to assessing spelling is based on the theoretical assumption that this allows for a comprehensively holistic, eclectic and multi-dimensional perspective on assessment and evaluation of spelling. As in this instance quantitative and qualitative aspects of data collection, data analysis and description are involved. Further, taking a mixed methods approach moves the assessor away from one-dimensional perspectives where quantitative sources which reflect absolute changes in spelling would not highlight how the pupil has moved from one pattern of spelling behaviour to another. However, qualitative aspects such as talk behavioural changes that, for example, reflect greater use of metacognitive processes such as reflective comments can be accounted for and inform pupil progress to a greater degree. This mixed methods approach echoes Brinton & Fujiki (2003) urge for practitioners to move away from single sources of information for evaluating change.

Arguably these behaviours could be counted and analysed quantitatively. However the focus here is not just on the product outcomes of an intervention but on exploring, in a descriptive way, the themes and patterns of behaviour that overtly or covertly imply development or changes in metacognitive knowledge or experience.

Method

Data was collected at non- treatment phase, immediately pre therapy, immediately post therapy and three months post therapy. Four dimensions of spelling ability were taken:- 1. Real word spelling inventory 2. Non- word spelling inventory 3. Spontaneous text level 4. Alphabetic knowledge Quantitative data sources were obtained from formal standardised assessment measures. Changes in spelling relating to qualitative measures were taken from questionnaire and video data. The latter allowing direct observation and interpretation of overt and covert verbal and non-verbal actions and interactions. A caveat to this approach is the level of observer inference required. However, in the view of Robson (2002) ‘as the actions of behaviours of people are central aspects in virtually any enquiry, a natural and obvious technique is to watch what they do, to record this in some way and then describe, analyse and interpret what we have observed” (p308)

Expected Outcomes

From a stakeholder’s perspective, (parents’, teachers’ and pupils’) a focus on qualitative and quantitative change as opposed to quantitative alone allows for product and process outcome measures. Correctly spelt items are accounted for along with the strategies that the pupil is using, ultimately, resulting in a balance between product and process change. It is predicted that quantitative and qualitative approaches will acknowledge developmental changes that indicate how the pupil has progressed along cognitive and linguistic parameters of spelling. Data collected indicates that qualitative information provides greater depth and breadth of spelling change within a developmental framework. It is further hypothesised that a mixed methods approach to assessment and evaluation of spelling change will result in empowerment of stakeholders such as parents and teachers. Both of which are empowered and given a knowledge base on which to support and develop spelling changes. This is particularly important as parent’s they are often the primary support mechanism for pupil’s outside of the school context. From the pupil perspective it is predicted that they will focus on process and product aspects of spelling, resulting in greater generalisation and durability of learning.

References

Apel, K., J. J. Masterson, et al. (2004). Spelling assessment frameworks. Handbook of Language and literacy: Development and disorders. C. Addison Stone, E. R. Silliman, B. J. Ehren and K. Apel. London, Guildford press: 644-60. Bishop, D. V. M. and B. Clarkson (2003). "Written Language as a window into residual language deficits: A study of children with persistent and residual speech and language impairments. ." Cortex 39: 215-237. Brinton, B. and M. Fujiki (2003). "Blending Quantitative and Qualitative Methods in Language Research and Intervention." American Journal of Speech-Language Pathology 12: 165-171. Delinger, A. B. & N. L. Leech (2007). "Toward a Unified Validation Framework in Mixed Methods Research." Journal of Mixed Methods research 1(4): 309-332. Greene, J. C. (2006). "Toward a Methodology of Mixed Methods Social Inquiry." Research in the Schools 13(1): 93-98. Flavell, J. H. (1979). "Metacognition and Cognitive Monitoring: A New Area of Cognitive-Development Inquiry." American Psychologist 34(10): 906-911. Johnson, R. B., Onwuegbuzie, A., & Turner, L. (2007). "Toward a Definition of Mixed Methods Research." Journal of Mixed Methods research 1(2): 112-133. Leask, A. and F. Hinchliffe (2007). "The effect of phonological awareness intervention on non-word spelling ability in school-aged children: An analysis of qualitative change. ." Advances in Speech- Language Pathology 9(3): 226-241. Robson, C. (2002). Observational methods. (p.309-345). Real World Research. Oxford. Blackwell Publishing Tashakkori, A. and J. W. Creswell (2007). "The New Era of Mixed Methods." Journal of Mixed Methods research 1(1): 3-7. Vellutino, F. R., Fletcher, J. M. Snowling, M. J. & Scanlon, D. M. (2004). "Specific reading disability (dyslexia): what have we learned in the past four decades?" Association for Child Psychology and Psychiatry. 45(1): 2-40. Yin, R. K. (2006). "Mixed Methods Research: Are the Methods Genuinely Integrated or Merely Parallel." Research in the Schools 13(1): 41-47.

Author Information

Newcastle University
Speech, and Language
Newcastle upon Tyne

Update Modus of this Database

The current conference programme can be browsed in the conference management system (conftool) and, closer to the conference, in the conference app.
This database will be updated with the conference data after ECER. 

Search the ECER Programme

  • Search for keywords and phrases in "Text Search"
  • Restrict in which part of the abstracts to search in "Where to search"
  • Search for authors and in the respective field.
  • For planning your conference attendance, please use the conference app, which will be issued some weeks before the conference and the conference agenda provided in conftool.
  • If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.