Developing Activities for Assessing “Wicked Sustainability Problem”-Literacy – A Collaborative Action Research Approach

Author(s):

Johanna Lönngren(presenting / submitting)Magdalena Svanström

Conference:

ECER 2016

Network:

30. Environmental and Sustainability Education Research (ESER)

Format:

Paper

Session Information

30 SES 11 C, Assessing ESE and Programs

Paper Session

Time:

2016-08-25

17:15-18:45

Room:

NM-Theatre L

Chair:

Elsa Lee

Contribution

The world faces increasingly complex and “wicked” challenges to sustainability. Such wicked sustainability problems (WSPs) lack definite solutions and require holistic approaches that integrate many different perspectives from local and global stakeholders, as well as environmental considerations (Lönngren & Svanström 2015a; Rittel & Webber 1973; Seager et al 2012). Examples of current WSPs are climate change, resource scarcity, global health problems, and pollution. WSPs are not only particularly difficult to deal with; they also present serious threats to sustainability. Therefore, one of the main goals of environmental and sustainability education (ESE), especially in higher education, should be to prepare students to "holistically and integratively understand and address WSPs while considering the normative context of sustainable development", an ability which we will call “WSP-literacy” (c.f. Lönngren & Svanström 2015b).

WSP-literacy requires a different set of skills compared with solving non-wicked problems (Jonassen et al 2000; Kitchener 1983; Schraw et al 1995). Jonassen et al (2000) suggest that dealing with wicked problems requires, for example, an ability to reconcile conflicting goals, multiple forms of problem representation, and multiple solution methods. Such skills are not sufficiently in focus in many higher education programs.

The overall aim of our project was to develop concrete, practical approaches to ESE that can support (engineering) students’ development of WSP-literacy. To identify how ESE research could contribute to engineering ESE practice related to WSP-literacy, we conducted semi-formal interviews with four engineering ESE practitioners. The interviews were structured around the results from previous empirical work on how engineering students approach WSPs (authors, forthcoming). In the interviews, the educators identified a lack of adequate assessment activities as one of the most important factors that currently limits their ability to teach WSP-literacy.

King and Kitchener (1994) suggest that widely used, traditional and general approaches to assessment are inappropriate for assessing WSP-literacy. These approaches include for example non-wicked problem solving tasks, multiple-choice tests, and other approaches that require students to provide definite answers. King and Kitchener suggest that such approaches would be in conflict with the very nature of complex real-world problems. Consequently, new and different approaches to assessment are necessary for assessing WSP-literacy.

We formulated two specific aims for our project:

Design assessment approaches for WSP-literacy that are readily applicable to (engineering) ESE practice
Contribute to theoretical understandings of what characterizes useful and functional assessment approaches in ESE in general and for WSP-literacy specifically

We used constructive alignment theory (Biggs 1996, 2014) as a theoretical basis for working with aim 1 because this theory is widely known and accepted among engineering educators. Constructive alignment theory assumes that assessment activities not only serve to track student learning, but that they actually influence student learning and that they therefore should not be treated as an afterthought in designing educational programs. Rather, the educator should first define intended learning outcomes, then assessment activities, and finally prepare teaching materials that match the intended learning and assessment.

Constructive alignment is sometimes criticized as an instrumentalist approach to education. ESE might benefit more from a complete shift away from skill development and assessment, to a more holistic approach to educating the whole person and developing each person’s unique potential. Maybe it would be better to focus on realizing the opportunities of each unique educational context, rather than attempting to find generalized descriptions of skills and ways of assessing these skills. We acknowledge these limitations and wish to encourage and support efforts to genuinely transform the educational system. However, we believe that such genuine change will take time; in the meantime, we hope that our contribution can provide support to quickly improve ESE practice within the current system.

Method

To address the lack of adequate assessment activities for WSP-literacy, we have chosen to work together with ESE practitioners. We held three workshops in different settings. The first workshop was held at a local seminar with ESE practitioners in 2014. This was a mixed group of 27 engineering and non-engineering educators, most of which had extensive experience in working with ESE in higher education. During the 90-minute workshop led by the first author, the participants engaged with the concept of WSPs and WSP-literacy, and worked in seven groups to formulate intended learning outcomes for WSP-literacy. At the end of the workshop, each group presented their results. As a preparation for the second workshop, we (the authors) consolidated, clarified, and structured the suggestions from the groups in the first workshop. This resulted in a matrix of 22 learning outcomes, which we used as a basis for developing assessment activities in the second and third workshop. The second workshop was held at a local pedagogical conference at a technical university in 2015. In this workshop, five participants worked in two groups for approximately one hour. Each of the groups first read and discussed the provided learning outcomes. They were then asked to choose one learning outcome and design an assessment activity for that specific outcome. The groups took two very different approaches to the task, which provided an interesting starting point for workshop 3. The third workshop will be held at an international conference for engineering education in summer 2016. The workshop will be part of the program of the environmental education division at the conference and we expect most participants to be engineering ESE practitioners. A maximum of 24 participants is accepted for the workshop. This workshop will last for three hours. Thus, there will be more time for general discussions about WSPs, WSP-literacy, and assessment of WSP-literacy in engineering education. Most of the workshop, however, will again be devoted to group work on designing specific assessment activities for one (or a few) learning outcomes at a time. Finally, we (the authors) will again consolidate the results and describe the most powerful examples that emerged from the workshops. By comparing the examples with each other, and with traditional and generic approaches to assessment, we aim to identify salient characteristics of useful and functional assessment in ESE in general and for WSP-literacy specifically.

Expected Outcomes

The concrete results from our project include the above-mentioned matrix of intended learning outcomes and a set of examples for suitable assessment activities for WSP-literacy. The learning outcomes are presented in a two-dimensional matrix. In this matrix, learning outcomes are characterized as pertaining to an understanding of (a.) the concept of sustainable development, (b.) the problem situation in WSPs, and (c.) possible courses of action for addressing WSPs. ILOs were further categorized according to what kinds of learning outcomes they represent. For this purpose, a category system for different types of ILOs was developed, which builds on existing descriptions by the Swedish National Agency for Higher Education (Högskoleverket 2010, 2012), the UNESCO (2010), and Bloom’s taxonomy (Bloom et al 1956; Krathwohl 2002). The category system for types of learning outcomes that was developed includes five items: (1.) knowledge and comprehension, (2.) skills (application of knowledge), (3.) advanced thinking processes (analysis, synthesis, evaluation), (4.) attitudes and values (approaches, valuation), and (5.) action competence. The examples of assessment activities are described as detailed and concretely as possible, including descriptions of the characteristics of the intended student group for each activity and the educators’ descriptions of why this activity would be functional and useful to assess a particular aspect of WSP-literacy. We expect that ESE practitioners will find these results valuable and actionable, not only in engineering education, but in a range of disciplines of higher education that work with addressing complex environmental and societal problems. Finally, we draw conclusions about what may be critical features of the suggested assessment activities. We hope that these conclusions can guide further development of activities for a range of specific educational contexts, as well as further research into what characterizes “good” assessment in ESE.

References

Authors (2015). Paper presented at an international conference on engineering education. Authors (forthcoming). Paper under review for publication in Research in Science Education. Biggs, J. B. (1996). Enhancing teaching through constructive alignment. Higher Education, 32, pp. 1-18. Biggs, J. B. (2014). Constructive alignment in university teaching. HERDSA Review of Higher Education,  1, pp. 5-22. Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy of  educational objectives: The classification of educational goals. Handbook I: Cognitive domain. New York:  David McKay Company. Högskoleverket (2010). Rapport 2010:22 R. Högskoleverkets system för kvalitetsutvärdering 2011-2014.  Available at: http://hsv.se/download/18.4afd653a12cabe7775880003715/1022R-system-kvalitetsutv.pdf Högskoleverket (2012). Beslut om riktlinjer för val av mål vid utvärdering av utbildningar som leder till  generell examen (omgång 3–2012). Reg.nr 12-4013-10. Available at: http://www.uk-  ambetet.se/download/18.197eccc1140ee238b58b27/12-4013-10-beslut-mal-generell-examen.pdf Jonassen, D. H. (2000). Toward a Design Theory of Problem Solving. Educational Technology and Research Development , 48 (4), pp. 63-85. King, P. M., & Kitchener, K. S. (1994). Developing Reflective Judgment. San Francisco, CA: Jossey-Bass. Kitchener, K. S. (1983). Cognition, Metacognition and Epistemic Cognition: A three-level model of  cognitive development. Human Development, 26, pp. 222-232. Krathwohl, D. R. (2002). A Revision of Bloom's Taxonomy: An Overview. Theory Into Practice, 41:4,  pp. 212-218. Lönngren, J., Ingerman, Å., & Svanström, M. (in press). Avoid, Control, Succumb, or Balance: Engineering Students' Approaches to a Wicked Sustainability Problem. Research in Science Education. Lönngren, J., & Svanström, M. (2015a). Assessing "Wicked Sustainability Problem"-Literacy in Engineering Education. Proceedings of the 122nd ASEE Annual Conference & Exhibition, Seattle. Lönngren, J., & Svanström, M. (2015b). Systems thinking for dealing with wicked sustainability problems: beyond functionalist approaches. Proceedings of the 7th International Conference on Engineering Education for Sustainable Development, Vancouver, Canada. Rittel, HWJ, & Webber, MW. (1973). Dilemmas in a General Theory of Planning. Policy Sciences, 4, pp. 155-169. Schraw, G., Dunkle, M. E., & Bendixen, L. D. (1995). Cognitive Processes in Well-Defined and Ill-Defined Problem Solving. Applied Cognitive Psychology, 9, pp. 523-538. Seager, T, Selinger, E, & Wiek, A. (2012). Sustainable Engineering Science for Resolving Wicked Problems. Journal of Agricultural Environmental Ethics, 25, pp. 467-484. UNESCO (2010). Teaching and Learning for a Sustainable Future: a multimedia teacher education  programme. Retrieved from: http://www.unesco.org/education/tlsf/mods/theme_d/mod24.html

Author Information

Johanna Lönngren (presenting / submitting)

Chalmers University of Technology

Applied IT, Division of Engineering Education Research

Göteborg

Magdalena Svanström

Chalmers University of Technology, Sweden

Search the ECER Programme

Search for keywords and phrases in "Text Search"
Restrict in which part of the abstracts to search in "Where to search"
Search for authors and in the respective field.
For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.