This study addresses the language of the school subject mathematics and the aim is to investigate the potential of multimodal resources to express meaning in textbooks and digital teaching materials. An emphasis is in the analysis laid on the distinction between subject specific and everyday multisemiotic register. 

Language used in teaching materials in mathematics is often multisemiotic, which means that various semiotic resources such as natural language, symbolic notation, and images are used together. These semiotic resources have different potential to express meaning (Schleppegrell, 2007; Lemke, 1990; Unsworth, 1997; Abel & Exley, 2008). Natural language is argued to be a very poor resource for formulating for example quantity, continuous co-variation, and gradation (Lemke, 1998) and therefore there is also a need for other resources to express meaning in mathematics. When various semiotic resources are used together in a text, the text can express both more and other things, compared to the use of the different semiotic resources separately, a phenomenon referred to as meaning multiplication (Lemke, 1998). This multiplicativity of meaning is possible since in a multisemiotic text, the different semiotic resources contribute differently to the text, and the meaning afforded by one resource can modulate the meaning afforded by another resource.

In mathematics education today these various semiotic resources are extensively used, both in print and on computer screens. Images together with natural language and mathematical notation are used as resources in teaching, in order to strengthen the student’s conceptual knowledge (Brenner, Herman, Ho & Zimmer, 1999). During the 20^th century the presence of images in mathematics teaching materials has increased (Dimmel & Herbst, 2015), but most often, students get no education about the role and function of images (Kress & van Leeuwen, 2006). Lemke (2000) emphasizes the importance of deepening the understanding about the role of different semiotic resources. Such an understanding is also required by a student to master a subject, as part of the content knowledge since representations have such an intrinsic role in the subject mathematics. It is therefore of importance to find out more precisely how various semiotic resources are used in school mathematics, and if these resources are used differently in different kinds of teaching materials.

To learn more about the semiotic resources used in teaching materials in school mathematics the current study adopts a social semiotic theoretical perspective (see e.g., Kress and van Leeuwen, 2006; O’Halloran, 2007). This perspective provides tools to investigate both how aspects of language, such as various semiotic resources, are used in acts of communication, and at the same time analyze how these chosen forms of language express and thus offer meaning to the reader in different ways (see e.g. Knain, 2005). The backbone of the study is an analysis focusing on the three metafunctions: the interpersonal, ideational and textual function (Halliday & Matthiessen, 2014). The inclusion of all three metafunctions make it possible to highlight different semantic perspectives of interest both in relation to research about mathematics texts and for teaching.

A qualitative analysis is used to thoroughly understand how different textual means are used in mathematics teaching material and which meaning that is offered to the reader. A sample of mathematics texts that introduces proportionality is analyzed. In this study both digital and printed teaching material are referred to as text. The texts are of different types to obtain a breadth and to enable a comparison between texts with different purposes. Both teaching materials used in the primary school (11 years old) and teaching materials intended for a sub-group of upper secondary school students (16 years old) are analyzed. These two types of texts are analyzed to illuminate how the language resources are used for students at different levels in the education. Both printed texts and digital teaching materials are also analyzed. Digital teaching material and printed mathematics text have different means available; in the digital media sound, film and interactive elements may be utilized. Those elements are important to include in the analysis to represent the whole composition of representations offered by the teaching material. However, in the initial analysis of the digital teaching materials only texts and images has been analyzed in detail, something that has been taken into account in relation to these preliminary results. The final analysis will be complemented with a multimodal analysis focusing on interactive elements, film, and sound in the digital teaching material (see O’Halloran, 2011); focusing on how these elements interact with other components of the material. The analytical tool has been developed based on previous work by Kress and van Leeuwen (2006), O’Halloran (2005, 2007), and Royce (2007). An emphasis is in the analytical tool put on its ability to distinguish between subject specific and everyday multisemiotic register, and on how particular affordances of the semiotic resources are used. In this study subject specific register is defined as language with a technical meaning or used with a technical meaning in the subject of mathematics, language that is not part of the everyday language for the intended readers. The analysis of digital and printed teaching material is conducted at two levels; first the natural language and the images are analyzed separately. Thereafter the intersemiotic complementarity of the texts is analyzed. The inclusion of both levels of analysis is motivated since the different elements of the text both function separately and together as a whole to express meaning.

The study will contribute with knowledge about the potential of multimodal resources to express meaning in textbooks and digital teaching materials. The preliminary analysis show that by taking advantage of the affordances of the different semiotic resources the ideational meaning can be expressed in a coherent way. Such an example can be found in a text introducing proportionality with an example. Speed is illustrated by a cartoon image representing a moving person and an explanatory sentence. Thereafter the mathematical content is presented utilizing subject-specific expressions, in natural language and in a graph. The cartoon is however included in the graph, which gives coherence to the text by making relations between the everyday content and the subject specific more pronounced. An opposite to this use of images are when images are used in a solely illustrative purpose. Another result is that in the textbook as well as in the digital material for year 5, there is an evident personal voice expressed by persons present in the images or by proper names or personal pronouns in the written text. These features serves as subjects in the texts as well as in the images. The personal voice can signal to the reader that mathematics is something that concerns people's everyday lives. In the analyzed material for upper secondary school, personal voice is used more sparsely. Instead, the mathematical objects functions a subjects, both in the texts and in the images. In this way, a distance between the reader and the mathematical content is expressed. In summary the results from the analysis of material written for different student groups, both in print and digital media, contribute with examples of how the different semiotic resources can function as meaning making resources.

Abel, K. & Exley, B. (2008). Using Halliday’s functional grammar to examine early years worded mathematics texts. Australian Journal of Language & Literacy. 31(3), 227-241. Brenner, M. E., Herman, S., Ho, H-Z., & Zimmer, J. M. (1999). Cross National Comparison of Representative Competence. Journal for Research in Mathematics Education, 30(5), 541–557. Dimmel, J. K., & Herbst, P. G. (2015). The semiotic structure of geometry diagrams: How textbook diagrams convey meaning. Journal for Research in Mathematics Education, 46(2), 147-195. Halliday, M., & Matthiessen, C. (2014). Halliday's introduction to functional grammar (4.th ed.). Abingdon, Oxon; New York: Routledge. Knain, E. (2005). Identity and genre literacy in high-school students' experimental reports', International Journal of Science Education, 27:5, 607 - 624. Kress, G. (2005). Gains and losses: New forms of texts, knowledge, and learning. Computers and Composition 22, 5–22. Kress, G. & van Leeuwen, T. (2006). Reading images. The grammar of visual design. 2nd edition. London: Routledge. Kress, G. (2010). Multimodality: A social semiotic approach to contemporary communication. Milton Park, Abingdon, Oxon: Routledge. Lemke, J. L. (1990). Talking Science: Language, Learning, and Values. Ablex, Norwood, N.J. Lemke, J. (1998). Multiplying meaning. Visual and verbal semiotics in scientific text. In J. R. Martin, & R. Veel. Reading images. London: Routledge. (SIDOR 87-113) Lemke, J. (2000). Multimedia literacy demands of the scientific curriculum. Linguistics and Education, 10 (3), 247–271. O'Halloran, K. (2005). Mathematical Discourse: Language, symbolism and visual images. London: Continuum. O’Halloran, K. (2007). Systemic functional multimodal discourse analysis (SF–MDA) approach to mathematics, grammar and literacy. In A. McCabe, M. O’Donnell, and R. Whittaker (eds). Advances in Language and Education. London: Continuum. O’Halloran, K. (2011). Multimodal Discourse Analysis. In K. Hyland and B. Paltridge (eds) Companion to Discourse. London and New York: Continuum. Royce, T. (2007). Intersemiotic complementarity: a framework for multimodal discourse analysis. In T. Royce, & W. Boucher. New Directions in the Analysis of Multimodal Discourse. (pp. 63-109). New York: Routledge.