Along with useful information Internet also includes a lot of biased information resources presented through multimodal means aiming at promoting some ideological or commercial interests (e.g., Kammerer et al., 2015). Critical evaluation of argumentation of various online resources is thus important. Both middle and high school (Wineburg & McGrew, 2016), upper secondary (Kiili et al., 2008) and college students (McClure & Clink, 2009; Wineburg & McGrew, 2016) have been found to have difficulties to critically evaluate online information. Thus, there is a call for increasing our knowledge on how students evaluate online resources including various information presented both through written text and multimodally.

Students need argumentative analysis skills not only for evaluating the credibility of online resources but also when they engage in tasks in which they have to integrate information from multiple sources to produce written essays. The better argument analysis skills students have the better argumentation they also produce (Preiss, Castillo, Flotts & San Martino, 2013). Further, in collaborative learning situations students are often required to negotiate meanings of various source texts, discuss different viewpoints, and weight various optional solutions for problems encountered in order to form their own understanding on issues (Kiili, Laurinen, Marttunen, & Leu, 2012; Newell, Beach, Smith & VanDerHeide, 2011). These skills are particularly important in academic settings when students are expected to write argumentative syntheses in which various competing arguments are integrated (c.f. Cuevas et al., 2016). Previous studies have shown that students at both secondary (Cambliss, 1995; Sole, Miras, Castells, Espino & Minguela, 2013) and undergraduate levels (Howard et al., 2010; Segev-Miller, 2004) have difficulties in tasks in which they are asked to integrate information from multiple sources to produce written documents. Further, Utriainen et al. (2017) found that also university applicants have difficulties to analyze argumentative texts. As many upper secondary school students continue to university studies the importance of investigating their skills to evaluate argumentation of online resources is highlighted.

The aim of this study was to evaluate upper secondary school students’ skills to analyze arguments and related justifications of two different internet resources. Additionally, the study investigated how well gender, grades in mother tongue and health sciences, and parents’ educational level predicted students’ performance in the two argument analysis tasks.

Research questions were the following:

1. What is the level of upper secondary school students’ argument analysis skills with regard to a blog text and a multimodal video?

2. Do students’ gender, grades in mother tongue and health sciences, and parents’ educational level predict their argument analysis skills?

This study is a sub study of a larger research project investigating upper secondary school students’ argumentative online inquiry skills (ARONI project). Participants of the study were 404 Finnish students (16 - 20 years) from seven schools. The students performed an online task including two different online resources, a blog text and a You Tube video, which dealt with vaccination of children from opposite perspectives. The blog text presented several potential disadvantages caused by vaccinations ending up with a conclusion that children should not be vaccinated. The YouTube video, by contrast, discussed the benefits of vaccination and the conclusion was that it is wise to vaccinate children. Arguments presented for and against vaccination in these texts were classified into various argumentation perspectives (e.g. Effectiveness of vaccinations). The blog text included four and the YouTube video two argumentation perspectives. Students’ task was to identify from both texts the author’s main position concerning vaccination of children and the most important justifications used to support it. Students’ responses to the task for both the video and the blog text were scored from 0 to 4 points. No points were given if the student had not recognized the position of the author at all. Students’ responses were scored from 1 to 4 points depending on a) whether the position they had identified represented a correct conclusion of the source text (i.e. Everybody should vaccinate their children/It is not safe to vaccinate children) or was only descriptive by nature (i.e. Vaccinations do not cause autism/Vaccinations are dangerous) and b) how many argumentation perspectives they had identified from the source to support the position. For the blog text one point was given if the student’s response represented a descriptive claim either not justified at all or justified with one justification, or if the response represented a correct conclusion without any justifications. For the video, in turn, 1 point was given if the response represented a descriptive claim and no justifications were presented to support it. In the other extreme of the rating scale, full points (score 4) were given if the response included a correct conclusion justified either from three argumentation perspectives (blog text) or from two perspectives (video). Structural Equation Modeling was used for testing how gender, parents’ educational level, and students’ grades in mother tongue and health science predicted the students’ performance in the argument analysis tasks.

For the YouTube video only 1 % of students did not recognize the position of the video at all (0 points), whereas 21 % of them recognized the position at a descriptive level (1 point). Almost a third (31.3 %) of students earned two points as they had either identified the position of the video at a descriptive level and justified it from one perspective, or had recognized the correct conclusion but had not presented any justifications. Furthermore, 36.6 % of students earned 3 points, and 10.1 % earned 4 points from the task. For the blog text, the distribution of students’ points was as follows: 0 points, 13 %; 1 point, 32.7 %; 2 points, 42.3 %; 3 points, 20.9 %; and 4 points, 2.8%. Students performed better in the task for the video (M = 2.3, SD = 1.0) than for the blog text (M = 1.9, SD = 0.8) (t = 7.14, df = 395, p < .001). The tested model predicting students’ performance fit to the data well: χ²(1) = 1.82, p = 0.178; RMSEA = .05. The model explained 5.3 % of students’ performance in analyzing the blog text showing two significant predictors, grades in mother tongue (β = .12, p = .046) and health science (β = .13, p = .020). None of the variables explained students’ performance in analyzing the YouTube video. In general, evaluation of argumentation of online resources turned out to be a difficult task for the students. This result is supported with similar findings in previous studies (Kiili et al., 2008; Wineburg & McGrew, 2016). Furthermore, the study suggests that upper secondary schools do not yet provide enough support for students to learn online inquiry skills. Thus, teaching of these skills should be emphasized in the upper secondary school curriculum.

Chambliss, M. J. (1995). Text cues and strategies successful readers use to construct the gist of lengthy written arguments. Reading Research Quarterly, 30(4), 778−807. Cuevas, I., Mateos, M., Martín E., Luna. M., Martín, A., Solari, M., Gonzáles-Lamas, J & Martínez I. (2016). Collaborative writing of argumentative syntheses from multiple sources: The role of writing beliefs and strategies in addressing controversy. Journal of Writing Research, 8(2), 205-226. doi: 10.17239/jowr-2016.08.02.02 Howard, R. M., Serviss, T. & Rodrigue, T. K. (2010). Writing from sources, writing from sentences. Writing and Pedagogy, 2.2, 177−192. Kammerer, Y., Amann, D., & Gerjets, P. (2015). When adults without university education search the Internet for health information: The roles of Internet-specific epistemic beliefs and a source evaluation intervention. Computers in Human Behavior, 48, 297–309. Kiili, C., Laurinen, L., & Marttunen, M. (2008). Students evaluating Internet sources: From versatile evaluators to uncritical readers. Journal of Educational Computing Research, 39(1), 75–95. Kiili, C., Laurinen, L., Marttunen, M. & Leu, D. 2012. Working on understanding during collaborative online reading. Journal of Literacy Research, 44(4), 448–483. McClure, R. & Clink, K. (2009). How do you know that?: An investigation of student research practices in the digital age. Portal: Libraries and the Academy, 9(1), 115–132. Wineburg, S. & McGrew, S. 2016. Why students can’t google their way to the truth. Education Week, 36(11), 22, 28. Newell, G., Beach, R., Smith, J. & VanDerHeide, J. (2011). Teaching and learning argumentative reading and writing: A review of research. Reading Research Quarterly, 46(3), 273–304. Preiss, D. D., Castillo, J. C., Flotts, P. & San Martino, E. (2013). Assessment of argumentative writing and critical thinking in higher education: Educational correlates and gender differences. Learning and Individual Differences, 28, 193–203. doi:10.1016/j.lindif.2013.06.004 Segev-Miller, R. (2004). Writing from sources: the effect of explicit instruction on college students’ processes and products. L1−Educational Studies in Language and Literature, 4(1), 5−33. Sole, I., Miras, M., Castells, N., Espino, S. & Minguela, M. (2013). Integrating information: An analysis of the processes involved and the products generated in a written synthesis task. Written Communication, 30(1), 63−90. Utriainen, J., Marttunen, M., Kallio, E., & Tynjälä, P. (2017). University Applicants’ Critical Thinking Skills: The Case of the Finnish Educational Sciences. Scandinavian Journal of Educational Research, 61(6), 629−649. doi:10.1080/00313831.2016.1173092.