Session Information
06 SES 06 A, Discussing and Producing Multimodal Representations in Open Learning
Paper Session
Contribution
In recent decades, technology has substantially modified how adolescents generate content and interact socially through various digital platforms (OECD, 2021). The digital era has transformed the traditional notion of textual composition (Danielsson & Selander, 2021) since, in digital environments, messages are produced by non-linear and interactive elements composed of various semiotic modes, such as images, sounds, and verbal text (Flewitt, 2011). Therefore, the digital content composition must be conceived from a multimodal perspective (Jewitt, 2003) to understand how different semiotic modes contribute to meaning-making according to communicative purposes (Bezemer & Jewitt, 2018). Multimodal composition in digital environments has been studied as Digital Multimodal Composing (DMC), defined as the production of digital content through various semiotic modes to negotiate meanings with a real or imaginary audience (Smith et al., 2021). DMC has been studied from Social Semiotic theory (Kress, 2009), which states that individuals select specific semiotic resources depending on modes’ potential—or affordance—to communicate meanings (Kress, 2009) according to sociocultural factors, discursive genre, and the relationship with the audience (Smith, 2017). DMC has also been studied under the theories of multiliteracies (Cope & Kalantzis, 2009) since multimodal composition processes respond to new pedagogical practices related to the digital age, design thinking skills and students' authentic contexts (Cope & Kalantzis, 2009; Miller, 2013). Empirical studies in secondary school contexts have shown that DMC processes positively affect the development of higher-order skills (Knobel, 2017), such as creation, creativity, critical thinking, collaborative work, and problem-solving. Likewise, DMC promotes the development of linguistic skills (Vandommele et al., 2017), social skills (Mills, 2009), metacognitive strategies (Hung, 2019), digital empathy (Jiang & Gao, 2020), and metalanguage to consciously use semiotic resources in multimodal texts production (Unsworth & Mills, 2020). Although DMC practices positively impact the development of learning and skills, some studies have shown that young people do not always have the skills needed to generate content in digital environments (Bennett et al., 2008; Dalton, 2012) due to the scarcity of teaching strategies to evaluate and guide DMC tasks (Purcell et al., 2013). Consequently, it is relevant to conceive DMC processes not only as a means to achieve learning purposes but as a skill that requires the development of pedagogical strategies to help students create digital content through new ways of representation (Jocius, 2013).
Recent literature reviews on DMC tasks with secondary school students have examined students' processes and perceptions in DMC contexts (Li & Akoto, 2021), the relationship between DMC and translanguage with bilingual and multilingual students (Pacheco et al., 2022), and teaching strategies to work on DMC in the classroom (Smith et al., 2021). These literature reviews allow to know how the implementation of DMC in secondary education has been empirically analyzed; however, these studies focus mainly on language learning (L1-L2), and lees on other disciplines, such as scientific areas, in which the use of multimodality is essential to represent phenomena and elements that can hardly be explained only by verbal language (Bergey et al., 2015). This literature review aims to expand understanding of DMC processes, incorporating different school disciplinary areas. Therefore, this literature review seeks to answer the question: What do empirical studies report on DMC implementation in secondary education from diverse disciplinary areas? Knowing the practices teachers and students carry out during DMC processes will contribute to understanding the pedagogical strategies for developing multimodal language and its impact on digital literacies.
Method
A literature review was conducted using the Web Of Science (WOS) database, in which the following codes were used: TS=(“digital multimodal compos*” OR “multimodal writing” OR “multimodal compos*” OR “digital compos*” OR “digital multimodal” OR “multimodal codemeshing” OR multimodality) AND TS=(“middle school” OR “junior high” OR “high school” OR “secondary”). Combining these search codes resulted in 107 articles published between 2018 and 2023 (the search ended on September 6th, 2023). This review aimed to determine what the empirical evidence reports on DMC implementation from diverse disciplinary areas in secondary school education. A first screening was performed by analyzing the titles and abstracts (N=107). At this stage, theoretical studies, literature reviews, and studies focused only on teachers were rejected. Then, a second screening focused on full-text articles (N=77) dismissed studies according to the analysis object: multimodal artifacts, classroom interactions, and digital educational platforms. The final sample of documents for subsequent analysis consisted of 30 articles. Subsequently, the articles were described according to the research design, the country where studies were implemented, and the disciplines or themes related to DMC tasks. Open coding (Corbin & Strauss, 2002) was used to identify emerging elements. Next, axial coding (Corbin & Strauss, 2002) was used to gather the elements into thematic categories. The categories were divided into two phases: the first focused on the students' practices during the DMC processes, and the second focused on the teachers' practices during the DMC processes.
Expected Outcomes
The articles examined correspond mainly to qualitative studies, such as case studies, ethnographic studies, design-based research, and critical discourse analysis. Only three investigations used a mixed approach. The studies were mainly implemented in the United States, followed by China, South Korea, Singapore, Denmark, Finland, Canada, New Zealand, and Norway. Concerning the disciplines or themes, most of the DMC tasks were related to social topics, second language teaching, and, to a lesser extent, science-related areas. The analysis was divided into two parts. The first is related to DMC processes from the students' perspective, and the second is related to DMC processes from the teachers' perspective. In the first section, which focused on students, two categories emerged: (1) the factors influencing the selection of semiotic modes—emotions and identities, the type of content, and the relationship with the audience— 2) perceptions about DMC practices—DMC benefits and tensions. Two categories emerged in the second section, which focused on teachers: (1) strategies for teaching DMC—emphasis on technical aspects and metalanguage teaching—(2) perceptions about DMC processes—benefits and challenges of DMC in teachers' educational contexts.
References
Bennett, S., Maton, K., & Kervin, L. (2008a). The “digital natives” debate: A critical review of the evidence. British Journal of Educational Technology, 39(5), 775–786. https://doi.org/10.1111/j.1467-8535.2007.00793.x Bezemer, J., & Jewitt, C. (2018). Multimodality: A guide for linguists. In L. Litosseliti (Ed.), Research Methods in Linguistics (Bloomsbury) Cope, B., & Kalantzis, M. (2009). “Multiliteracies”: New Literacies, New Learning. Pedagogies: An International Journal, 4(3), 164–195. https://doi.org/10.1080/15544800903076044 Dalton, B. (2012). Multimodal composition and the common core state standards. Reading Teacher, 66(4), 333–339. https://doi.org/10.1002/TRTR.01129 Danielsson, K., & Selander, S. (2021). Multimodal Texts in Disciplinary Education: A Comprehensive Framework. (Springer) Flewitt, R. (2011). Bringing ethnography to a multimodal investigation of early literacy in a digital age. Qualitative Research, 11(3), 293–310. https://doi.org/10.1177/1468794111399838 Jewitt, C. (2003). Reshaping Learning. New Technologies and Multimodality. International Journal of Learning, 10. Jiang, L., & Gao, J. (2020a). Fostering EFL Learners’ Digital Empathy through Multimodal Composing. RELC Journal, 51(1), 70–85. https://doi.org/10.1177/0033688219898565 Li, M., & Akoto, M. (2021). Review of Recent Research on L2 Digital Multimodal Composing. International Journal of Computer, 11, 1–16. http://www.igi-global.com/authorseditors/titlesubmission/newproject.aspx Hung, S. (2019). Creating Digital Stories: EFL Learners’ Engagement, Cognitive and Metacognitive Skills. Kress, G. (2009). A social-semiotic theory of multimodality. In Multimodality. A Social Semiotic Approach to Contemporary Communication (pp. 54–78) Miller, S. M. (2013). A research metasynthesis on digital video composing in classrooms: An evidence-based framework toward a pedagogy for embodied learning. Journal of Literacy Research, 45(4), 386–430. https://doi.org/10.1177/1086296X13504867 Mills, K. A. (2009). Multiliteracies: Interrogating competing discourses. Language and Education, 23(2), 103–116. https://doi.org/10.1080/09500780802152762 Pacheco, M. B., Smith, B. E., Combs, E., & Amgott, N. A. (2022). Translanguaging within multimodal composition products and processes: A systematic review. Pedagogies, 17(4), 389–407. https://doi.org/10.1080/1554480X.2022.2139256 Purcell, K., Buchanan, J., & Friedrich, L. (2013). The Impact of Digital Tools on Student Writing and How Writing is Taught in Schools/Teachers-technology-and-writing. In JULY (Vol. 16). http://pewinternet.org/Reports/2013/Teachers-technology-and-writing Smith, B. E. (2017). Composing across modes: a comparative analysis of adolescents’ multimodal composing processes. Learning, Media and Technology, 42(3), 259–278. https://doi.org/10.1080/17439884.2016.1182924 Smith, B. E., Pacheco, M. B., & Khorosheva, M. (2021). Emergent Bilingual Students and Digital Multimodal Composition: A Systematic Review of Research in Secondary Classrooms. Reading Research Quarterly, 56(1), 33–52. https://doi.org/10.1002/rrq.298 Unsworth, L., & Mills, K. A. (2020). English language teaching of attitude and emotion in digital multimodal composition. Journal of Second Language Writing, 47. https://doi.org/10.1016/j.jslw.2020.100712
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.