Perspectives For Defining Performance Indicators Of Teaching Skills Of Student Teachers For Providing Formative Feedback Through Learning Analytics

Author(s):

Edgar Krull(presenting / submitting)Äli Leijen(presenting)Marieke van der Schaaf Bert Slof

Conference:

ECER 2015

Network:

10. Teacher Education Research

Format:

Paper

Session Information

10 SES 04 C,

Paper Session

Time:

2015-09-09

09:00-10:30

Room:

209.Oktatóterem [C]

Chair:

Patrick Geffard

Contribution

The aim of this study is to analyse possibilities for the performance-based formative evaluation of student teachers’ teaching skills in their field practice by using learning analytics (LA). LA is conceived as a computer-based instructional system for documenting assessment results, providing specified feedback on the achievement of expected objectives and for providing suggestions for corrective and further learning activities in the framework of this study (e.g. Clow, 2013). The targeted analysis is part of the international project “Workplace-based e-Assessment Technology for Competency-based Higher Multi-professional Education” WATCHME (2014). An analysis of research literature on LA applications reveals that application of computer-based approaches at this level of sophistication calls for a valid and reliable definition of performance-based indicators of teaching skills and for taking into account possibilities of statistical modelling of the progress in teaching skills on the basis of quantified data on indicator performances. So far LA possibilities for capturing progress in teaching competence in an integrative way is limited as any algorithm or method…” applied in it ”…is reductive by nature in that it simplifies reality to manageable set of variables” (Greller and Drachsler, 2012, p. 50).

As application of LA calls for describing of teaching competence through performance indicators at high specification level not needed in the conventional procedures used for assessing teaching competence, an objective of this study is to learn possibilities for providing validity and reliability of the assessment procedures of the teaching competence based on these indicators. To this end an analysis of relevant research literature was carried out on studies of identifying teaching competence, applying outputs of these studies for stating teacher standards, and on identifying performance-based indicators for assessing teaching skills that are used for compiling assessment matrixes. Furthermore, the application of LA procedures calls for fast documenting of evidences on teaching activities in order to provide just-in-time feedback on the progress made. From this point of view possibilities for effective usage of portfolios and application of the concept core practices for focussing on the most essential performance indicators, were investigated. Also, issues of adapting performance-based assessment matrixes of teaching competence developed in the context of one country to condition of another country were analysed and exemplified.

For achieving the aim of this study answers to four questions were searched:

(1) To what extent is good teaching identifiable when working with different students and in different contexts?

(2) How valid and reliable are the theory-based characteristic of effective teaching and teacher standards?

(3) What are the options for compiling valid and reliable matrixes of measurable indicators of student teachers’ teaching competences that can be used for LA-based feedback on students’ progress?

(4) What are the options for using portfolios for timely collection of evidence on teaching and transferring them into quantified data on teaching competences?

Method

This study is based on a literature review targeting: (1) defining, modelling, and measuring teaching competences; (2), describing teacher standards and competence requirements used for teacher certification, and (3) defining performance-based indicators for formative assessment of student teachers in their school practice by using LA procedures. To this end databases Academic Search Complete, E-journals, ERIC, PsycARTICLES, PsycINFO, and Teacher Reference Center were repeatedly searched for academic publications in the last 20 years (1995-2015). The main keywords, used for searching in combination with restricting term “teacher education”, were: teaching competences, competence requirements, professional standards, teaching standards performance-based assessment, professional development, competency-based instruction, teaching practice, performance indicators meta-analysis, core practices, learning analytics. In addition journals Teaching and Teacher Education, Journal of Teacher Education, European Journal of Education, Educational Leadership were researched for finding works not covered by used keywords. The analysis started with requirements for the quality of entry data needed for effective use of LA procedures and proceeded with defining performance-based indicators of student teachers’ teaching competence in their field practice. The analysis of perspectives for defining performance-based indicators of teaching skills took place in three consecutive phases. (1) It started with an analytic survey of research on defining effective teaching. (2) This was followed by an analysis of four practice-oriented models of teaching competences created in the light of theoretical studies like teacher standards or competence requirements that have undergone more or less thorough validation attempts. These validation procedures included analysis of former research (e.g. Looney, 2011), reaching practitioners’ consensus (e.g. Dutch teacher competence requirements; SBL, 2004), empirical research (NBPTS standards; NBPTS, 2014), and deducting more specific standards from validated general standards (e.g. INTASC standards; The Council … 2013). In the phase three our analysis focused on the possibilities of defining performance-based indicators of teaching skills in the format of assessment matrixes that were coherent with the theoretical knowledge on effective teaching and with practice-oriented teacher standards. Also options for collecting data on teaching performances by using portfolios and for avoiding excessive decomposing of teaching into atomistic performances by focussing on relevant learnable parts for student teachers as foresees the concept of core practice were analysed from the perspective of effective application of LA.

Expected Outcomes

This study is expected to prepare the ground for optimizing student teachers’ professional learning in school practice by applying LA for providing adaptive formative evaluation of their progress. The application of LA procedures for a valid and reliable assessment of student teachers’ progress in learning to teach is more demanding regarding the quality of entry data (being quantified) for trustful assessment of teaching than it would be in the case of assessment by a supervisor or mentor who can rely on their integrative view of student teacher performance and on their intuition. Therefore, more attention should be paid on the validation of the assessment matrixes of performance indicators with defined performance levels. More specifically, the study is expected to provide solutions at least for three circles of problems. Firstly, to uncover possibilities for validating performance-based assessment matrixes of teaching skills that satisfy the needs of using LA procedures. This includes clarifying of the potential role of research, validated teacher standards and comparisons of LA-based assessment with expert ratings for teaching competence. Secondly, the study is expected to inform about the need and ways of adjusting assessment matrixes compiled for LA applications in one country to the conditions of another country. Thirdly, to identify effective ways for operative collecting performance-based data on teaching that would enable providing student teachers with just-in time feedback on their progress in acquiring teaching skills. Our analysis leads to a conclusion that theoretical knowledge on manifestations of teacher competence and validated teacher standards provide only coordinating ideas to be followed for compiling matrixes for performance-based assessment. The specification of performance-based indicators of teaching skills follows the same principles as defining of specific performance-based learning objectives on the basis general aims or goals of education in the objective-driven approaches to instruction (e.g. Gronlund, 1988; Araisian, 1994).

References

Airasian, P. W. (1994). Classroom assessment. Second Edition. New York: McGraw-Hill. Clow, D. (2013). An overview of learning analytics. Teaching in Higher Education. Aug2013, Vol. 18 Issue 6, p 683-695. Greller, W.; Drachsler, H. (2012) . Translating Learning into Numbers: A Generic Framework for Learning Analytics. Educational Technology & Society, 15(3), 42-57. Gronlund, N. E. (1988). How to construct achievement tests. Fourth ed. Englewood Cliffs, NJ: Prentice-Hall, Inc. Looney, J. (2011). Developing High-Quality Teachers: teacher evaluation for improvement. European Journal of Education, (46) 4, 440–455. NBPTS. (2014). Certificate Areas. Retrieved from http://www.nbpts.org/certificate-areas SBL (2004). Competence requirements. Teachers. Retrieved from http://www.bekwaamheidsdossier.nl/cms/bijlagen/SBL_Introduction.pdf The Council of Chief State School Officers (2013). Interstate Teacher Assessment and Support Consortium InTASC Model Core Teaching Standards and Learning Progressions for Teachers 1.0: A Resource for Ongoing Teacher Development. Washington, DC: Author. http://www.ccsso.org/Documents/2013/2013_INTASC_Learning_Progressions_for_Teachers.pdf WATCHME: Workplace-based e-Assessment Technology for Competency-based Higher Multi-professional Education (2014). Retrieved from http://cordis.europa.eu/project/rcn/189095_en.html

Author Information

Edgar Krull (presenting / submitting)

University of Tartu

Institute of Educational Science

Tartu

Äli Leijen (presenting)

University of Tartu, Estonia

Marieke van der Schaaf

Utrecht University, The Netherlands

Bert Slof