Predictive Utility of Short Videos For Teaching Performance Of Preservice Teachers

Author(s):

Ruth Geer(presenting / submitting)Bruce White(presenting)Graham Hardy

Conference:

ECER 2016

Network:

10. Teacher Education Research

Format:

Paper

Session Information

10 SES 12 C, Perspectives on Education and Training

Paper Session

Time:

2016-08-26

09:00-10:30

Room:

NM-F106

Chair:

Milosh Raykov

Contribution

Preservice teacher education in Australia is under scrutiny and is high on the political agenda as the Australian Government recognises the importance of having well educated and knowledgeable teachers who can provide high quality education to our young people (Craven et al., 2014). The Teacher Education Ministerial Advisory Group (TEMAG) produced a report that identified five themes, including improved and structured practical experience for teacher education students and robust assessment of graduates to ensure classroom readiness. The report also endorsed the recommendation of a revision of the accreditation of initial teacher education to ensure high quality graduates with the required skills and knowledge to impact on student learning (TEMAG, 2015). Wright (2015) and Heinz (2013) in their investigations identified that academic success was not necessarily related to performance on placement. Such research highlights the complexity of teaching and the importance of more consistent and rigorous assessment in identifying the range of qualities evidenced by a successful teacher.

A short self-shot video was considered as one element that could identify preservice teachers who are likely to be successful in their professional experience placement and those who may need some additional support. The focus of this study then is to investigate whether a short video has a predictive utility of future teaching performance.

Effective communication is an essential element in learning to teach and preservice teachers also need to develop these interpersonal skills (Bower, Cavanagh, Moloney, & Dao, 2011). Teachers require a range of communication skills, such as listening, interrupting, writing, as well as presentation skills, such as voice projection, body language and gesture (Bower et al., 2011, p. 312). The presentation of concepts and delivery of instructions is important in influencing motivation, and student outcomes (McCroskey, Richmond, & Bennett, 2006). Also Mazer (2013) found that communication behaviours such as clarity and body language can stimulate emotional interest and engage students. Videos provide a useful tool in analysing both the verbal and non-verbal behaviours of preservice teachers.

Videos or video portfolios have been used in teacher education programs to assess preservice teachers’ growth in recognising effective teaching strategies (Wiens, Hessberg, LoCasale-Crouch, & DeCoster, 2013); and in demonstrating the implementation of essential concepts into their teaching practice (Admiraal, Hoeksma, van de Kamp, & van Duin, 2011); as a valid method for assessing teacher competencies (Admiraal & Berry, 2015). With effective communication seen as an essential skill for good teaching, the production of a short video was seen as a tool for selecting preservice teachers who may become successful teachers, while also giving reliable judgements on who may require additional support.

Method

This study describes a number of processes that were used in the research design to explore how a brief video by first year Master of Teaching (Secondary) preservice teachers might be used to identify teaching performance. Prior to commencing their teacher education courses, preservice teachers were asked to prepare a one and a half minute video of a talk that they would present to 14 year olds on “things you can do to help you with your learning”. Using the Adaptive Comparative Judgement web-based program, a group of six teacher educators compared 83 videos submitted by the preservice teachers. The use of an adaptive comparative judgement ranking system, which provides a sophisticated ranking mechanism for the comparison of like objects, was explored as a possible approach to rank preservice teachers, providing an indicator of future teaching performance. Adaptive Comparative Judgement, which was derived from Thurstone’s (1927) discovery that people are unreliable when making absolute judgements but are more dependable for relative judgements, requires educators to compare the work of two students, based on identified criteria, deciding which is better. From many such comparisons a ranking scale is created showing the relative quality of students’ performance. Based on the criteria of who was best at communicating their ideas in a clear, concise and well sequenced manner using appropriate language many comparisons of the videos were made to determine their preference for one of two videos in a pair. A high reliability in excess of 0.96 was achieved with each video being judged between eleven and thirteen times. However validity was probably much lower as the suitability of the criteria was still being explored. Following on from this the researchers wanted to test the stability of the ranking order using the same tool and the same educators using the criteria of whose talk was more likely to engage and interest the learner. This provided a different rank order with a correlation between the two rankings of 0.61 which indicated that the criteria may need to be refined. These rankings were compared with preservice teachers who failed, experienced difficulties or withdrew from their placement (generally because they experienced problems or realised that teaching was not for them) over their first two placements. The intention was to see if these rankings would identify problematic students and those who needed additional support.

Expected Outcomes

The rankings showed that the students in the top half all performed well. Surprisingly those in the lower third also completed satisfactory placements and it was those preservice teachers around the middle of the ranking that seemed to struggle. An outcome to date confirms the importance of good assessment design and the identification of criteria upon which to assess the videos. Further investigation of suitable criteria is yet to be explored. Further examination on the predictive value of the videos is proceeding using adaptive comparative pairs to examine preservice teacher performance based on their final reports. The statements from their mentors relating to their professional practice, which is one of the Australian Graduate Teacher Standards will be compared. A recognised limitation in assessing the statements is that some mentors are better at articulating the experience than others. The rankings from analysing the mentor statements from these reports will be correlated against the two video rankings and once again compared with those preservice teachers who failed, withdrew or experienced difficulties. An outcome of the study highlights the importance of assessment criteria. However the tool works best when criteria are simple and it becomes problematic with more than one criteria. Finally, conclusions will be drawn on the predictive use of videos in identifying potential teaching performance for this particular cohort of preservice teachers. Further research on other cohorts is also underway to further ascertain reliability of videos as an indicator of teaching performance.

References

Admiraal, W., & Berry, A. (2015). Video narratives to assess student teachers’ competence as new teachers. Teachers and Teaching(ahead-of-print), 1-14. Admiraal, W., Hoeksma, M., van de Kamp, M.-T., & van Duin, G. (2011). Assessment of teacher competence using video portfolios: Reliability, construct validity, and consequential validity. Teaching and teacher education, 27(6), 1019-1028. Bower, M., Cavanagh, M., Moloney, R., & Dao, M. (2011). Developing communication competence using an online Video Reflection system: pre-service teachers' experiences. Asia-Pacific Journal of Teacher Education, 39(4), 311-326. Craven, G., Beswick, K., Fleming, J., Fletcher, T., Green, M., Jensen, B., . . . Rickards, F. (2014). Action Now: Classroom Ready Teachers. Retrieved from http://www.studentsfirst.gov.au/teacher-education-ministerial-advisory-group Heinz, M. (2013). Tomorrow’s teachers—selecting the best: An exploration of the quality rationale behind academic and experiential selection criteria for initial teacher education programmes. Educational Assessment, Evaluation and Accountability, 25(2), 93-114. Mazer, J. P. (2013). Associations among teacher communication behaviors, student interest, and engagement: A validity test. Communication Education, 62(1), 86-96. McCroskey, J. C., Richmond, V. P., & Bennett, V. E. (2006). The relationships of student end-of-class motivation with teacher communication behaviors and instructional outcomes. Communication Education, 55(4), 403-414. Teacher Education Ministrial Advisory Group. (2015). Action Now: Classroom ready Teachers - Australian Government Response. Retrieved from Canberra: https://docs.education.gov.au/system/files/doc/other/150212_ag_response_-_final.pdf Wiens, P. D., Hessberg, K., LoCasale-Crouch, J., & DeCoster, J. (2013). Using a standardized video-based assessment in a university teacher education program to examine preservice teachers knowledge related to effective teaching. Teaching and teacher education, 33, 24-33. Wright, V. J. (2015). Is ATAR useful for predicting the Success of Australian Students in Initial Teacher Education? Australian Journal of Teacher Education, 40(9), 1.

Author Information

Ruth Geer (presenting / submitting)

University of South Australia

School of Education

Mawson Lakes

Bruce White (presenting)

University of South Australia, Australia

Graham Hardy