Students Transitioning Between Personal And Educational Use Of Information And Communication Technologies

Author(s):

Anthony Jones(presenting / submitting)

Conference:

ECER 2015

Network:

16. ICT in Education and Training

Format:

Paper

Session Information

16 SES 04 A, The Transition to Digital Resources

Paper Session

Time:

2015-09-09

09:00-10:30

Room:

3005. [Main]

Chair:

Adriana Gewerc

Contribution

IT use in school classrooms has been researched and reported on for at least four decades (Tamin, et al., 2011). Recent large scale international studies have collected data relating to computer access, student perceptions of computer use for learning, socio economic status of students and parents, information literacy levels, as well as IT related knowledge and skills. It is claimed that the International Computer and Information Literacy Study (ICILS) was “the first international comparative study that examines students’ acquisition of computer and information literacy” (De Bortoli et al., 2014, p.3). Around 60 000 students participated and they were assessed on their ICIL (information and computer literacy) knowledge skills at home and at school within two broad strands: collecting and managing information, and producing and exchanging information. Among the findings reported were links between computer related skills and knowledge and computer access at home and school, and with the socio-economic background of the participants.

Australian data collected in 2011 (ACARA, 2012) indicated that 90% of Grade 6 students used computers one or more times each week at home compared with 85% at school. This is in line data from the 2013 ICILS study (De Bortoli et al., 2014) where the figures for Year 8 were 87% at home and 81% at school. Comparatively, the figures for the Czech Republic were 96% at home and 60% at school, and for Poland 96% at home and 79% at school. The ICILS means were 87% at home and 54% at school. This study also reported that the ratios of students per school computer were 13 for the whole cohort, 3 for Australia and 10 for the Czech Republic.

One consequence arising from increased IT use for learning is the interest in replacing the current paper-based mode of assessment with an online mode. When this was suggested in 2011 by the education department in the Australian state of Victoria, there was concern among teachers and parents from schools where students had limited computer access at home or at school.

Despite the results from large surveys, many teachers are not convinced the results were an accurate indication of computer use and knowledge at their schools. As part of this concern an Assistant Principal invited the author to collaborate in an investigation into student use and perceptions of IT in one Australian primary school. Two separate research studies were planned and implemented. The first involved all students in grades 4 and 6, about 175 in total, and investigated the perceived and actual differences of the students between paper-based and computer-based testing [assessment] in mathematics. The second study involved one class of 26 grade 4 students. The aim of this research was to investigate whether the mathematical content knowledge of the students could be strengthened and measured through their creation of a multimedia artefact and subsequent discussion and writing about the artefact to a designated audience. The school was considered to be in an area of low socio-economic status, with more than 85% of students coming from a non-English speaking background, and almost no parents having a recognised tertiary qualification.

Method

Both studies employed a qualitative mixed methods approach that included the video-recording of classroom and computer room activities, focus group interviews and discussions, and the collection of mathematics tests and multimedia artefacts. The first study involved the students in grades 4 and 6 undertaking two mathematics tests, one paper-based and the other computer-based. Both tests were developed and normed by the Australian Council for Educational Research (ACER) and were claimed to be normatively equivalent. Data collected for analysis included results of the two tests, together with video recordings of each class in a computer room undertaking the computer-based test and a few days later in their classroom participating in an informal focus group discussion and unstructured interviews. The paper-based and the computer-based test results were compared with each other and then with the students’ results from the national mathematics testing (NAPLAN) from the previous year. In addition, because of the enormous discrepancy between results from the paper-based and the computer-based tests, approximately two weeks later the students re-took the computer-based test, but this time in a paper-based format. The second study was managed by the classroom teacher with the researcher observing some classes and, several weeks after the data collection period, attending a demonstration where students displayed and explained their multimedia artefacts to parents and other teachers. Data collected and analysed consisted of the on-paper storyboards each student produced prior to using the nominated software and the digital stories that were produced. In addition several informal focus group discussions were held in the classroom in an attempt to gauge student beliefs and attitudes about the project. Among other things this study explored the transition for these students from playing multimedia games in which no educational content was clear, to designing a multimedia story that focused on nominated mathematical content. This was in line with the definition of computer information literacy as “the ability to use computers to investigate, create and communicate in order to participate effectively at home, at school, and in the workplace and in society” (Fraillon, Schultz & Ainley, 2013, p.17).

Expected Outcomes

A number of surprising and disturbing results were obtained in the first study, with perhaps the most significant being the poor perceptions expressed by both students and teachers concerning the concept of online testing. These perceptions could be summarised as the existence of a dichotomy – paper based tests were real and important while online tests were like a game and were therefore not important. The video-recordings of the online tests showed that very few students made any attempt to use the paper and pen provided for calculating or sketching, unlike what was observed when the tests were paper based. Student results from the two modes of testing were disparate, with clear evidence that while results from the paper-based test correlated with results from the 2010 Australian national testing in mathematics, the online test scores were some two deciles lower. As a follow up, several weeks later the students re-took the online test, but this time in a paper based format. The results were much closer to the original paper-based test and to the 2010 national test. The second study indicated that the Grade 4 students were quite capable of using a new piece of software to produce a mathematically based story. However there were some issues relating to how such artefacts could and should be assessed. The paper-based storyboard each student prepared was corrected by the teacher with an emphasis on any problems or errors in the mathematics and language. But when these were transformed into multimedia artefact there were many spelling and grammatical errors, perhaps partly reflecting the non-English speaking background of the students. A discussion with other teachers at the school about assessing the artefacts showed many different opinions, especially relating to whether they could be assessed for not only IT, but also English and mathematics.

References

Australian Curriculum, Assessment and Reporting Authority (ACARA). (2012). National Assessment Program— ICT Literacy Years 6 and 10 Report. Sydney: ACARA. De Bortoli, L., Buckley, S., Underwood, C., O’Grady, E. and Gebhart, G. (2014). Australian students’ readiness for study, work and life in the digital age. Retrieved from http://www.acer.edu.au/aus-icils Jones, A. & Truran, L. (2011). Paper and online testing: Establishing and crossing boundaries. Refereed paper in J. Wright (Ed) Proceedings of the Australian Association for Educational Research Conference, Hobart Australia. ISSN 1324-9320. Fraillon, J., Schulz, W., & Ainley, J. (2013). International Computer and Information Literacy Study Assessment Framework. Amsterdam: International Association for the Evaluation of Educational Achievement (IEA). Organisation for Economic Cooperation and Development (OECD) (2013). PISA 2012 results: Excellence through equity: Giving every student the chance to succeed. (Volume II). Paris: OECD Tamim, R., Bernard, R., Borokovski, E., Abrami, P. & Schmid, R. (2011). What forty years of research says about the impact of technology on learning: A second-order meta-analysis and validation study. Review of Educational Research, 81(1), pp. 4–28.

Author Information

Anthony Jones (presenting / submitting)

University of Melbourne

International Centre for Classroom Research