Session Information
30 SES 09 C, Assessment and Other Ideas
Paper Session
Contribution
Sustainability challenges are complex and "wicked"(Lönngren & van Poeck, 2021), making the understanding of sustained, effective and self-determined action for sustainability equally intricate. Consequently, researchers have focused on advancing various behavioural models (e.g. Lülfs & Hahn, 2014; Zabel, 2005).
Despite progress in Environmental and Sustainability Education (ESE), the development of measurement tools to assess its outcomes remains limited. Most existing scales still rely on models of knowledge, attitude, and simplistic behaviour change (Kollmuss & Agyeman, 2002), with few addressing conscious action or action competencies. Additionally, they often focus on environmental rather than broader sustainability issues and rely on Classical Test Theory (CTT) (Kim, 2019).To address these gaps, this research developed scales covering sustainability issues, measuring conscious, self-determined action, and employing the Rasch model to overcome CTT’s limitations.
This study aims to develop measurements essential for empowering self-determined action, including procedural knowledge, self-efficacy, locus of control, and action. The study introduces the Sustainability Actualisation Questionnaire (SAQ), comprising five scales: Procedural Knowledge, Self-Efficacy, Internal Sustainability Locus of Control, External Sustainability Locus of Control, and Action. This instrument is one of the first to be validated using Rasch modeling in the field of environmental and sustainability education.
The study employed a four-phase mixed-method design: a conceptual literature review, interviews, expert review, a pilot study, and scale administration. The items for the SAQ were developed through a comprehensive review of existing instruments and literature. Initially, the instrument was pilot-tested with 46 secondary school students, followed by normality, reliability, and Guttman structure analysis. A revised version was then administered to 244 students from 17 schools across Victoria and South Australia, with the results undergoing Rasch analysis, and reliability testing.
The final SAQ consists of 59 items across five scales, with responses recorded on a four-point Likert scale, ranging from strongly agree to strongly disagree. All five scales met the requirements of the Rasch model, including invariance of items, appropriate category ordering, and differential item functioning. Moreover, the scales demonstrated strong reliability. Notably, all scales were correlated, suggesting a need for further research to examine their interrelationships.
The results demonstrate that these scales can inform research-based practices by exploring how environmental and sustainability education can enhance students' procedural knowledge, self-efficacy, locus of control, and action. These scales can also be adapted for use in other cultural contexts with appropriate modifications. Further studies are needed to refine the constructs and develop scales suitable for diverse cultural and educational settings.
Method
This study employs an exploratory multiphase mixed-method design, combining qualitative and quantitative phases. This approach, commonly used in survey development (McMillan & Schumacher, 2010), enables triangulation of findings and enhances result fidelity by leveraging the strengths of different research methods (Tashakkori & Teddlie, 2003). The four-phase model adopted here, unlike typical two-phase designs, includes intermediate stages for refining the survey (Creswell & Plano Clark, 2018). The study encompasses four main phases, further divided into nine sequential and concurrent research activities. These phases were shaped by various research design sources (Clark & Watson, 1995; DeVellis, 2017; Gehlbach & Brinkworth, 2011; Specter, 1992). A key feature of this design was its flexible, iterative approach, blending literature review, empirical data collection, and theoretical development. The data collection methods were adjusted based on findings from each phase, ensuring continuous synthesis of results. This multi-stage approach, combined with literature and theoretical work, was vital for ensuring the validity and reliability of the scales. Phase I involved a literature review and semi-structured interviews to explore how locus of control (LOC) is understood theoretically and how it could be operationalized for Australian adolescents. It also examined potential connections between LOC, self-efficacy, procedural knowledge, and action. Unlike previous studies on Environmental LOC, which did not include qualitative phases in instrument development, this research prioritized understanding LOC and sustainability from participants’ perspectives. Previous ELOC instruments often relied on existing behavioural theories or oversimplified the LOC-action relationship, while this study’s qualitative phase advanced theoretical understanding, leading to the creation of a novel framework. In Phase II, the findings from Phase I guided the initial development of a questionnaire. This phase included item generation, initial item reduction, question formatting, and expert review. Phase III involved pilot testing the draft scale with 46 students, followed by two focus group interviews. The results of Phase III contributed to testing initial hypotheses and refining the theoretical framework and questionnaire for Phase IV. In Phase IV, the questionnaire was administered to 221 students, and the data were analyzed to assess its validity using the Rasch model. The Rasch analysis process allowed for evaluation and reflection on the theoretical framework developed earlier. Additionally, statistical analyses, including T-tests, ANOVA, Kruskal-Wallis H tests, and correlation analyses, were conducted. Through these analyses and a review of earlier phases, the final questionnaire and theoretical framework were established.
Expected Outcomes
The Sustainability Action Questionnaire (SAQ) enables researchers to assess how educational programs influence students’ internal SLOC, procedural knowledge, self-efficacy, and conscious action through pre- and post-intervention studies. It also supports research on action competence and critical pedagogies (Stevenson, 2008), providing a valuable tool for improving sustainability education (Biasutti & Frate, 2016; Johnson & Manoli, 2010). This study found correlations between procedural knowledge, self-efficacy, LOC, and action. Further research could use structural equation modeling and confirmatory factor analysis to explore how these constructs mediate or moderate each other. The SAQ could also be used across different Australian states to examine response variations by region, school type, or sustainability education approaches. The Rasch model requires unidimensionality to differentiate research participants by a single latent variable. However, developing a unidimensional scale for a composite variable is challenging, as including multiple items for each attribute makes it harder to meet this requirement. While unidimensionality is crucial when distinguishing people based on one attribute, a more precise, multidimensional instrument may be preferable when assessing the nuances of an individual's sustainability-related competencies. Future research should determine when to prioritize precision over unidimensionality in different educational contexts. The SAQ allows participants to focus on sustainability topics of interest, but research priorities may shift over time and by region. For example, biodiversity loss is currently a critical issue (Steffen et al., 2015). Specific scales, such as a biodiversity or climate change mitigation questionnaire, could be developed for different research foci. Finally, these scales could be adapted for different age groups and cultural contexts to explore how SLOC varies across populations. This would help identify universal and culturally specific items, ensuring broader applicability.
References
Biasutti, M., & Frate, S. (2016). A validity and reliability study of the attitudes toward sustainable development scale. Environmental Education Research, 23(2), 214-230. https://doi.org/10.1080/13504622.2016.1146660 Clark, L. A., & Watson, D. (1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7(3), 309-319. https://doi.org/10.1037/1040-3590.7.3.309 Creswell, J. W., & Plano Clark, V. L. (2018). Designing and conducting mixed methods research (3rd ed.). SAGE. https://doi.org/(ybp)12989424 DeVellis, R. F. (2017). Scale development: Theory and applications (4th ed.). SAGE. https://doi.org/(Ybp)12747826 Gehlbach, H., & Brinkworth, M. (2011). Measure twice, cut down error: A process for enhancing the validity of survey scales. Review of General Psychology, 15(4), 380-387. https://doi.org/10.1037/a0025704 Johnson, B., & Manoli, C. C. (2010). The 2-MEV scale in the United States: A measure of children's environmental attitudes based on the theory of ecological attitude. The Journal of Environmental Education, 42(2), 84-97. https://doi.org/10.1080/00958964.2010.503716 Kim, M. (2019). Actualising sustainability: understanding and measuring the role of sustainability locus of control in educational contexts Kollmuss, A., & Agyeman, J. (2002). Mind the gap: Why do people act environmentally and what are the barriers to pro-environmental behavior? Environmental Education Research, 8(3), 239-260. https://doi.org/10.1080/13504620220145401 Lönngren, J., & van Poeck, K. (2021). Wicked problems: a mapping review of the literature. International Journal of Sustainable Development & World Ecology, 28(6), 481-502. https://doi.org/10.1080/13504509.2020.1859415 Lülfs, R., & Hahn, R. (2014). Sustainable Behavior in the Business Sphere:A Comprehensive Overview of the Explanatory Power of Psychological Models. Organization & Environment, 27(1), 43-64. https://doi.org/10.1177/1086026614522631 McMillan, J. H., & Schumacher, S. (2010). Research in education: Evidence-based inquiry (7th ed.). Pearson. Specter, P. E. (1992). Summated rating scale construction. SAGE. https://doi.org/10.4135/9781412986038 Steffen, W., Richardson, K., Rockström, J., Cornell, S. E., Fetzer, I., Bennett, E. M., Biggs, R., Carpenter, S. R., De Vries, W., & De Wit, C. A. (2015). Planetary boundaries: Guiding human development on a changing planet. Science, 347(6223), 1259855. https://doi.org/10.1177/0973408215600602a Stevenson, R. B. (2008). A critical pedagogy of place and the critical place(s) of pedagogy. Environmental Education Research, 14(3), 353-360. https://doi.org/10.1080/13504620802190727 Tashakkori, A., & Teddlie, C. (2003). Handbook of mixed methods in social & behavioral research. SAGE. Zabel, H. U. (2005). A model of human behaviour for sustainability. International Journal of Social Economics, 32(8), 717-734. https://doi.org/10.1108/03068290510608228
Update Modus of this Database
The current conference programme can be browsed in the conference management system (conftool) and, closer to the conference, in the conference app.
This database will be updated with the conference data after ECER.
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance, please use the conference app, which will be issued some weeks before the conference and the conference agenda provided in conftool.
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.