Giving Data back to Teachers and Students - Digital Student Feedback for Instructional Improvement

Author(s):

Stefan Ting Graf(presenting / submitting)Dorthe Carlsen

Conference:

ECER 2016

Network:

16. ICT in Education and Training

Format:

Paper

Session Information

16 SES 12, ICT in Educational Contexts

Paper Session

Time:

2016-08-26

09:00-10:30

Room:

OB-H1.49 (ALE 2)

Chair:

Stefan Ting Graf

Contribution

Based on the case of a developed and tested student feedback tool we present the results of our follow up studies in order to discuss critical issues of data-driven improvement of teaching and learning.

Teacher feedback for student learning has been in focus for a while (Hattie 2007). Similar systematic research on students’ feedback is either narrow or missing. Previous studies deal with student evaluations as part of the institutions quality standards or as method to identify exemplary teaching, teachers or schools. Other studies only focus on the students’ self-evaluation for learning outcomes or on students’ perception of teacher feedback (references). We assume that immediate and regular student feedback cycles are underestimated in relation to instructional improvement.

Research on data-driven improvement forms a similar picture. There are a tradition within school effectiveness research that in a rather sociological and organizational perspective deals with school leadership and learning outcomes (Leithwood & Louis 2012, Anderson, Leithwood & Strauss 2010). According to latest Swedish review on IT-based evaluation and feedback most studies focus on individual, most effective and automatic system-generated ways of feedback to the student (Hirsh/Lindberg 2015). The upcoming tradition of data-driven instructional improvement can more or less focus on learning outcomes or on the learning process (Robinson et al. 2009). We want to investigate the possibilities of data-driven instructional improvement by digital and systematic student feedback cycles.

For this purpose we designed, developed and tested an app-like tool, call Student Barometer (Danish: Elevbaro.dk) in relation to a national developmental and research project for Inclusion and differentiation in digital environments (Graf 2013). The tool makes it easy for teachers to collect student feedback quickly and easily on daily bases by mobile devices. The interface for the students is inspired by an analog feedback tool – The learning rating scale (Nissen 2012), and design as five stepless gliders between a positive and negative smiley. In this way the tool collects the students’ self-reflective statements on their engagement, feeling of being challenged, their need for help, their participation and task-consciousness. The interface for the teacher is divided in a sharing part for distributing the Barometer to the students and a data viewing part, where teachers can access different representations of the data. There are four basic views of the data: a bar chart and a box plot of the last rating of the whole class, development lines and box plots over multiple ratings.

The follow-up research project dealt with two interrelated research questions.

In which ways can the digital feedback tool help teachers to gain new attention to students and improve the differentiation of teaching (and learning)?
How can the Student Barometer be redesigned in terms of an smooth and functional tool.

From the beginning our stance is hat technology is the wrong driver (Fullan 2011). The feedback tool has to be seen in a broader, didactical concern. In order to investigate the teachers’ typical ways of dealing with student feedback we carried out a survey study with all teachers in the communality of Esbjerg in order to identify typical teacher practices in relation to student feedback. The survey was designed to provide answers to the following central questions: When, how and about what do teachers collect student feedback and for which purposes? How do teachers use the feedback and conclude on it?

Method

Our presentation will be based on the two following approaches. The follow-up of study of tool in use was designed as a heuristic and quasi-anthropological investigation in order to identify technical, didactical and organizational issues. Our qualitative analysis by Nvivo coding is based on interview with teachers and students in relation to two courses. Furthermore we include some video observations of the tool in use and video observations of knowledge sharing situations between teacher colleagues. In order to identify typical use of student feedback we design the survey study similar to the broader research project (Kølsen et al. 2014) The operationalization tries to avoid general statements and attitudes of the teachers by focusing on the reporting of the concrete use of feedback in the last course. Even though the survey with 429 teachers is not representative, we can describe some tendencies (Bryman, 2004 i Thisted 2011: 140). The statistical examination is based on simple frequency, bi- and multivariate analysis and a few index constructions.

Expected Outcomes

First: The survey study shows somehow contradictory results. Teachers think that student feedback is very important and they report that they have high confidence in the students’ judgment. But they rarely collect and use student feedback. Even less teachers involve digital tools for collecting feedback. There seem to be two patterns of dealing with it-based feedback. On one hand teachers that use IT to collect student feedback are significantly less likely to change the content of their teaching and conceive the use of student feedback as duty. Predefined digital ways of collecting student feedback tend to be carried out ritually. On the other hand teachers that develop their own schemes and procedures for students’ feedback are significantly more willing to change the content and the materials of teaching. It seems that teachers influence on the feedback tool is critical in order to facilitate meaningful changes. Based on the qualitative follow-up research we present an outline of broader and didactical approach for the use of digital feedback systems and data-driven improvement of instruction and the students’ well-being. In which ways do teachers integrate the Student Barometer in their teaching and the evaluation of teaching, and how do they change instruction and other pedagogical actions? The third part invites to a discussion of the possibilities, limitations and meaning of data-driven improvement of teaching and learning in relation to the Student Barometer. It is crucial to define who is collecting and using which kind of data for which purposes, and who owns the data and the procedures. Here also the student shows up. How does the student get a voice in all this?

References

Anderson, S., Leithwood, K., & Strauss, T. (2010). Leading Data Use in Schools: Organizational Conditions and Practices at the School and District Levels. Leadership and Policy in Schools, 9(3), 292–327. Fullan, M. (2011). Choosing the wrong drivers for whole system reform. Summary of Seminar Series Paper No. 204, Centre for Strategic Education. Graf, S. T. (2013). How can the Development of Digital Learning Environments make a Difference for Differentiation in Teaching? – An Intervention Study. IARTEM Textbooks and Educational Medie in a Digital Age, Ostrava, IARTEM. Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Hirsh, Å. og Viveca Lindberg (2015): Formativ bedömning på 2000-talet – en översikt av svensk och internationell forskning. Vitenskapsrådet Research Project on Inclusion and differentiation in digital environments at http://auuc.demonstrationsskoler.dk/ Kølsen, C., et al. (2014) Metoderapport i relation til baseline for demonstrationsskoleforsøg. http://pure.au.dk/portal/files/95993953/Bilag_B._Metoderapport_i_relation_til_baseline_for_demonstrationsskolefors_g.pdf Leithwood, K. & Louis, L.S. (2012): Linking Leadership to Student Learning. Jossey Bass. Nissen, P. (2012). "Hvordan måler man om eleverne lærer noget i skolen?" Økonomi & politik 85(2): 25-33. Robinson, V.; Hohepa, M. & Lloyd, C. (2009): School leadership and student outcomes: Identifying what works and why. Best Evidence Synthesis Iteration. Wellington: Ministry of Education, New Zealand.

Author Information

Stefan Ting Graf (presenting / submitting)

University College Lillebælt, Denmark

Dorthe Carlsen

University College Syddanmark, Denmark

Search the ECER Programme

Search for keywords and phrases in "Text Search"
Restrict in which part of the abstracts to search in "Where to search"
Search for authors and in the respective field.
For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.