Approaching complex multimodal phenomena in educational settings–Insights from theory

Digital media tasks are increasingly used in university settings to assess students’ scientific knowledge. The use of such tasks has generated the need for principled analytical approaches to better understand how meaning-making operates in these artefacts. However, there has been a proliferation of approaches to multimodality in recent years and as a result deciding which tools to use is challenging for analysts and for subsequent translation into pedagogy. Systemic functional approaches to multimodal phenomena have proved useful as we study the digital artefacts produced in tertiary science classrooms. These approaches are grounded in Halliday’s (1994) social-semiotic theories of language and influenced by Kress and van Leeuwen’s (2006) visual grammar. Here, we present insights from theoretical and methodological developments in systemic functional approaches to multimodal phenomena. We illustrate these insights with a student-generated digital media artefact, revealing the affordances of a systemic functional approach and the perspectives it offers on semiosis.