Session Information
99 ERC SES 08 F, Sociologies of Education
Paper Session
Contribution
There is a growing interest in conducting Systematic Reviews in education for both research purposes and evidence-based policy making. Education research is a critical domain that grants us valuable insights into the intricate processes of learning and teaching. Within this dynamic field, researchers employ a diverse array of methods and approaches to investigate a wide spectrum of educational facets, ranging from the dynamics of classroom environments and teacher-student interactions to the far-reaching consequences of education policies and practices on student outcomes (Creswell & Poth, 2019).
Typically, education research is categorized into two overarching paradigms: qualitative and quantitative research. However, it's worth noting that mixed-method research occupies a distinctive paradigm of its own. This comprehensive approach brings with it unique foundational perspectives on social reality and research, distinct ontological and epistemological viewpoints, and a set of axiologies and methodologies exclusive to its domain.
However, Systematic Reviews synthesising qualitative research evidence still pose theoretical and methodological challenges at all stages of the process (from the formulation of research questions to the evidence claim made by the authors).
In the field of education, the synthesis of qualitative studies within systematic reviews has long been a shared challenge. Education, inherently qualitative in nature, presents a complexity of variables that complicates the calculation of a straightforward combining effect size in meta-analysis (Borenstein, 2009). Moreover, the profound insights into educational settings and perceptions derived from the synthesis of qualitative studies hold immense value. Understanding not just whether a practice was successful, but why it was, offers a deeper perspective. However, the diversity in how qualitative research is conducted and reported poses significant challenges in synthesizing these findings (Sandelowski & Barroso, 2006).
While there's a prevailing belief that qualitative research can be systematically reviewed and synthesized, distinguishing methodically conducted qualitative research from those lacking rigor remains a necessity. Furthermore, a consensus on various aspects of the research process and its reporting still eludes the field (Garside, 2014).
Qualitative systematic reviews in Education provide valuable insights into the characteristics of knowledge claims made within the field. These reviews are designed to synthesize and analyse qualitative research studies to generate comprehensive and nuanced understandings of educational phenomena. The characteristics of knowledge claims in qualitative systematic reviews are distinct and supported by the review authors through rigorous methodology and transparent reporting. Critical reflection on facts and the interpretation of evidence lies at the core of all research, particularly when using research findings to guide policies and practices. Within the context of a systematic review, this process takes on added significance. Here, it is not only essential to deliberate upon the review methodology but also scrutinize the studies that have been incorporated into the review and dissect the resultant findings (Gough et al., 2017). This interconnected triplet – the research question, research methods, and research data – forms the linchpin of constructing knowledge claims within the purview of qualitative systematic reviews in the field of Education. Against this backdrop, this review aims at answering the following research question:
What are the characteristics of the knowledge claims made in qualitative systematic reviews in the field of education and how are they supported by the authors of the reviews?
a) conducting a systematic review of existing systematic reviews of qualitative studies in the education field with a focus on student, teacher and parents’ subjective experiences, beliefs, opinions and attitudes;
b) developing a comprehensive theoretical framework by integrating Toulmin's Argumentation Model and Gough's Claim Appraisal Framework (Gough, 2022; Toulmin, 1958) to identify methodological characteristics and reporting practices of qualitative systematic reviews in the education field.
Method
This review is preceded by an iterative protocol including detailed inclusion and exclusion criteria, a structured search process to locate and select relevant existing reviews, and a formal process to extract data. The screening process will be documented using the PRISMA Flow Diagram (Page et al., 2021) For the purposes of this study, only systematic reviews focused on qualitative studies that included the subjective experiences of students, teachers and parents were considered. Only primary or secondary education was taken into account. An initial search of ERIC database was undertaken. The search strategy used to construct clear and meaningful objectives was developed around three main concepts informed by the PCC framework (Pollock et al., 2023): Population: students, teachers, parents. Concept: students, teachers’ and parents’ subjective experiences (beliefs, attitudes, perceptions, opinions, experiences). Context: school environment, from primary to secondary education within the formal educational system where students engage in structured learning activities. Following the search, all identified citations have been collated into Zotero and then uploaded into Rayyan where duplicates were removed. The records in Rayyan were deemed eligible for inclusion if they met the following eligibility criteria. Inclusion criteria: 1. Must be a systematic review (a review and synthesis of existing primary research studies with reported methods) 2. The primary studies included in the systematic review should use qualitative methods (i.e investigate the views/ beliefs/ attitudes/ perceptions/ opinions/ experiences of participants using text / narrative/ speech as data). 3. The systematic review must use a qualitative method of synthesis. 4. The participants should be students, teachers, or parents in primary or secondary school settings (from grade 1 to 12). 5. The topic of the research should be education or learning broadly conceived. Exclusion criteria: 1. A review without methods and/ or primary research studies. 2. The primary studies included in the systematic review use quantitative methods (data is in the form of numbers). 3. The systematic review uses a statistical method of synthesis. 4. The participants are not students, teachers, or parents in primary or secondary school settings (from grade 1 to 12). 5. The topic of the research is health
Expected Outcomes
This research study is part of a PhD project which aims to explore the intricacies of 'knowledge claims' within the existing literature, particularly within qualitative reviews. To accomplish this goal, from a theoretical and conceptual perspective, an integration of Toulmin's model (Toulmin, 1958) with Gough's framework (Gough, 2022) will be enhanced. This synergistic approach will enable a comprehensive analysis of the characteristics inherent in these knowledge claims as they are portrayed in the body of qualitative research literature. The search strategy in ERIC yielded 335 studies, of which two were removed as duplicates. The remaining 333 studies were screened against the inclusion and exclusion criteria. The systematic review is still in progress, but the author expects to proceed simultaneously in two directions: (i) screening of studies for inclusion and (ii) construction of the framework for assessing the 'fit for purpose' of evidence claims by integrating Toulmin's model of argumentation and Gough's framework. This step is necessary to code the eligible studies (systematic reviews) that used qualitative research designs. These will be categorised into broader, higher order themes based on the integrated framework.
References
- Borenstein, M. (2009). Effect sizes for continuous data. In L. V. H. H. Cooper &. J. C. Valentine (Eds. ). (A c. Di), The handbook of research synthesis and meta-analysis (1–Book, Section, pp. 221–235). Russell Sage Foundation. - Creswell, J. W., & Poth, C. N. (2019). Qualitative inquiry & research design: Choosing among five approaches (4th ed.). SAGE Publication. - Garside, R. (2014). Should we appraise the quality of qualitative research reports for systematic reviews, and if so, how? Innovation: The European Journal of Social Science Research, 27(1), 67–79. - Gough, D. (2022a). Appraising Evidence Claims. Review of Research in Education, 45(1), 1–26. https://doi.org/10.3102/0091732X20985072 - Gough, D. (2022b). Appraising Evidence Claims. Review of Research in Education, 45(1), 1–26. https://doi.org/10.3102/0091732X20985072 - Gough, D., Oliver, S., & Thomas, J. (2017). An introduction to systematic reviews (2nd ed.). SAGE. - Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., (et al.) (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372. https://doi.org/10.1136/bmj.n71 - Pollock, D., Peters, M. D. J., Khalil, H., McInerney, P., Alexander, L., Tricco, (et al.) (2023). Recommendations for the extraction, analysis, and presentation of results in scoping reviews. JBI Evidence Synthesis, 21(3), 520–532. https://doi.org/10.11124/JBIES-22-00123 - Sandelowski, M., & Barroso, J. (2006). Handbook for synthesizing qualitative research. Springer publishing company. - Toulmin, S. (1958). The uses of argument. University Press
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.