Session Information
99 ERC SES 08 P, Research Capacity, Doctoral Education and Academic Careers
Paper Session
Contribution
Many doctoral students conducting qualitative research encounter difficulties selecting appropriate research methods and instruments. These demands are complicated by time constraints and inconsistent supervisory support. Although pilot studies are commonly recommended as a practical strategy for refining study designs, their role and value in phenomenological inquiry have received limited attention in the literature (Moustakas, 1994; Finlay, 2002). This proposal investigates the value of phenomenological pilot studies by reflecting on the researcher’s experiences in a mixed-methods doctoral project focused on teacher professional development.
"Pilot studies have useful functions in conducting qualitative research, but they have attracted scant attention in literature" (Kim, 2010, p. 191). For novice researchers, pilot studies can be a great way to work through uncertainties before committing to a full trial. They offer hands-on experience, close gaps in supervision or training, and shape a research identity (O'Cathain et al., 2015; Malmqvist et al., 2019). These small studies are useful for refining research designs and testing participant recruitment strategies (Lancaster et al., 2004; Kraemer et al., 2006).
Notwithstanding these benefits, reports documenting practical challenges and lessons from pilot studies remain scarce (van Teijlingen et al., 2001). Many researchers are unsure about how to measure the success of a pilot study. This leads to missed opportunities for improvement. This paper addresses this gap by proposing an evidence-based framework for assessing pilot study effectiveness in qualitative doctoral research.
The study examines the use of pilot studies in two distinct European educational contexts. Finland, which has a systematic approach to teacher professional development, has established a global reputation as a model educational system. Its research community has evolved from adopting international best practices to becoming "more a giving partner than a receiving one" (Sahlberg, 2021, p. 242). The Czech Republic has been strengthening its recent educational policies. This pairing enables the examination of how pilot study challenges manifest differently in a mature professional development system compared to a developing one. The cross-national aspect required considering linguistic and cultural factors during the pilot phase. This was especially relevant for communication norms related to recruiting teachers. This work informs European discussions about doctoral education and teacher professional development by addressing methodological preparation gaps in diverse doctoral training environments and aligning with EU policy priorities for strengthening teacher professional learning across member states (European Commission, 2020).
This study contributes to cross-national phenomenological pilot studies in European educational research, investigating how cultural, linguistic, and educational system differences impact the design and outcomes of pilot studies.This approach closes a gap in the current literature. The analysis reveals three key cross-cultural challenges: culturally specific recruitment trust patterns, linguistic bracketing requirements, and the influence of national professional development systems on teachers' professional identity. In addition to cross-cultural insights, this study redefines pilot studies as environments in which researchers cultivate their research identity.
The analysis demonstrates how pilot studies serve a compensatory pedagogical function in European doctoral programmes with limited formal methods training, a role overlooked in pilot study literature. Supervisors can enhance doctoral students' pilot study experiences by providing structured support through regular check-ins, targeted feedback, and peer collaboration, thereby strengthening both methodological rigour and researcher identity.
The following research question guided the analysis: How do phenomenological pilot studies function as sites of both methodological refinement and researcher identity development in doctoral mixed-methods research on teacher professional development?
Pilot study objectives are:
1. Develop and pilot core qualitative research procedures.
2. Test and refine participant recruitment strategies.
3. Design and refine the interview protocol through piloting.
4. Evaluate the feasibility of the transcription process and phenomenological
analysis approach.
5. Reflect on the pilot study and document the lessons learnt.
Method
This paper presents a phenomenological pilot study carried out from March to June 2024. As a qualitative component of a mixed-methods design, it focused on foreign language teachers' professional development in the Czech Republic and Finland, employing a phenomenological approach to explore participants' lived experiences (Moustakas, 1994; Creswell & Poth, 2023). The process involved developing the study framework, recruiting participants, designing and piloting an interview protocol, conducting two online interviews, and analysing the data. The researcher contacted participants using publicly available school email addresses. Through purposive sampling, one foreign language teacher from a lower-secondary school in each country was recruited (n = 2). This sample size is consistent with the pilot study purpose, which is to test procedures rather than achieve data saturation (van Teijlingen et al., 2001). Participants were the first respondents meeting inclusion criteria: lower-secondary foreign language teachers completed professional development in the preceding 12 months. They participated in 30–60-minute online interviews on MS Teams. The interview questions were revised using Hatch's (2023) guidelines. The data analysis used Creswell and Poth's (2023) Data Analysis Spiral, along with Moustakas's (1994) Stevick–Colaizzi–Keen method. The methodological reflection was based on several sources, such as a reflective journal, methodological memos, procedural documentation, transcription and analysis notes, supervisor feedback, and comparisons of initial and refined instruments (van Teijlingen & Hundley, 2001; Tracy, 2020; Creswell & Poth, 2023). The researcher reviewed documentation and grouped the entries based on procedural efficacy, skill enhancement, methodological modifications, resource considerations, and the researcher’s stance. Critical incidents, such as recruitment failures or problematic interview questions, were identified and analysed to determine their implications for the main study. Building on Tracy's (2010) quality criteria for qualitative research, this study proposes five dimensions for evaluating pilot study success: procedural viability, instrument quality, researcher competency development, problem identification, and resource efficiency. Together, these dimensions determine readiness for the main study. Rather than documenting 'lessons learnt', this framework gives doctoral researchers concrete criteria to determine whether their pilot has prepared them for the main study. While developed through a Finland-Czech comparison, this framework addresses common challenges across European doctoral programmes. The framework's adaptability in different contexts makes it a valuable tool. The evaluation focuses on (1) procedural effectiveness, (2) researcher skill development, (3) methodological adjustments and rationale, (4) time and resource implications, and (5) researcher positioning and bracketing challenges in phenomenological research (Tracy, 2020; Moustakas, 1994; Finlay, 2002, 2014).
Expected Outcomes
This pilot study reconceptualises phenomenological pilot studies as a stage for developing a methodological identity, where doctoral researchers transition from learning research methods to applying them. Procedurally, the pilot mitigated recruitment risks: generic email invitations produced one response over two weeks. This trial revealed teachers’ reluctance to respond to standardised requests. However, shifting to personalised messages increased response rates by 40%, saving eight weeks and preventing recruitment issues in the main study. The pilot exposed critical protocol weaknesses. Academically framed questions prompted participants to give “correct” answers, rather than explore their lived experiences. This echoes van Manen’s (1990) concept of the "colonisation of lived experience by abstract concepts". Rewording questions in teachers’ everyday language improved response depth. The cross-country comparison highlighted an important point: the Finnish participant used academic language, while the Czech participant struggled with theoretical terms, showing that linguistic bracketing must account for national differences in research literacy and necessitating country-specific interview guides. The pilot confirmed that phenomenological interviewing requires linguistic bracketing: researchers must bracket assumptions and academic language. Most significantly, the pilot enabled the researcher to develop “reflexive insider phenomenology". It became clear that bracketing personal teaching experience was neither possible nor desirable. Engagement with participants corresponded with Finlay's (2002; 2005) "reflexive embodied empathy", wherein professional knowledge enriches rather than distorts comprehension. The pilot created a space to examine this tension, develop skills in active listening, familiarity with MAXQDA, and balance insider knowledge with phenomenological openness. These insights extend discussions of pilot studies as procedural tools (Kim, 2010), showing how difficulties such as recruitment failures and protocol limitations serve as formative learning experiences rather than setbacks. This reframing reveals pilot studies' compensatory pedagogical function; in doctoral programs with limited formal methods training, they substitute for structured coursework, developing both methodological rigour and research identity.
References
Creswell, J. W., & Poth, Ch. N. (2023). Qualitative inquiry & research design: Choosing among five approaches (5th ed.). SAGE Publications. European Commission. (2020). Supporting teacher and school leader careers – A policy guide – Report. Directorate-General for Education, Youth, Sport and Culture. https://data.europa.eu/doi/10.2766/972132 Finlay, L. (2002). “Outing” the researcher: The provenance, process, and practice of reflexivity. Qualitative Health Research, 12(4), 531–545. https://doi.org/10.1177/104973202129120052 Finlay, L. (2005). “Reflexive embodied empathy”: A phenomenology of participant researcher intersubjectivity. The Humanistic Psychologist, 33(4), 271–292. https://doi.org/10.1207/s15473333thp3304_4 Finlay, L. (2014). Engaging Phenomenological Analysis. Qualitative Research in Psychology, 11(2), 121–141. https://doi.org/10.1080/14780887.2013.807899 Hatch, J. A. (2023). Doing qualitative research in education settings (2nd ed.). State University of New York Press. Kim, Y. (2010). The pilot study in qualitative inquiry: Identifying issues and learning lessons for culturally competent research. Qualitative Social Work, 10(2), 190-206. https://doi.org/10.1177/1473325010362001 Kraemer, H. C., Mintz, J., Noda, A., Tinklenberg, J., & Yesavage, J. (2006). Caution regarding the use of pilot studies to guide power calculations for study proposals. Archives of General Psychiatry, 63(5), 484–489. https://doi.org/10.1001/archpsyc.63.5.484 Lancaster, G. A., Dodd, S., & Williamson, P. R. (2004). Design and analysis of pilot studies: Recommendations for good practice. Journal of Evaluation in Clinical Practice, 10(2), 307–312. https://doi.org/10.1111/j..2002.384.doc.x Malmqvist, J., Hellberg, K., Möllås, G., Rose, R., & Shevlin, M. (2019). Conducting the pilot study: A neglected part of the research process? Methodological findings supporting the importance of piloting in qualitative research studies. International Journal of Qualitative Methods, 18, https://doi.org/10.1177/1609406919878341 Moustakas, C. (1994). Phenomenological Research Methods. Sage publications. O’Cathain, A., Hoddinott, P., Lewin, S., et al. (2015). Maximising the impact of qualitative research in feasibility studies for randomised controlled trials: guidance for researchers. Pilot and Feasibility Studies, 1, 32. https://doi.org/10.1186/s40814-015-0026-y Sahlberg, P. (2021). Finnish lessons 3.0: What can the world learn from educational change in Finland (3rd ed.). Teachers College Press. Tracy, S. J. (2010). Qualitative quality: Eight “big tent” criteria for excellent qualitative research. Qualitative Inquiry, 16(10), 837–851. https://doi.org/10.1177/1077800410383121 Tracy, S. J. (2020). Qualitative research methods: Collecting evidence, crafting analysis, communicating impact (2nd ed.). Wiley-Blackwell. Van Manen, M. (1990). Researching lived experience: Human science for an action sensitive pedagogy. State University of New York Press. Van Teijlingen, E. R., Rennie, A.M., Hundley, V., & Graham, W. (2001), The importance of conducting and reporting pilot studies: The example of the Scottish Births Survey, Journal of Advanced Nursing, 34(3), 289-295. https://doi.org/10.1046/j.1365-2648.2001.01757.x
Update Modus of this Database
The current conference programme can be browsed in the conference management system (conftool) and, closer to the conference, in the conference app.
This database will be updated with the conference data after ECER.
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance, please use the conference app, which will be issued some weeks before the conference and the conference agenda provided in conftool.
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.