Teachers Assessment in the European Space for Higher Education

Author(s):

Arturo Galán GonzálezIsabel Muñoz

Conference:

ECER 2009

Network:

11. Educational Effectiveness and Quality Assurance

Format:

Paper

Session Information

11 SES 08 A, Learning Outcome and Attitudes Towards Education

Paper Session

Time:

2009-09-30

08:30-10:00

Room:

HG, HS 46

Chair:

Jan A. Van Damme

Contribution

The evaluation of the quality of the university system is becoming increasingly important in Spain. Faculties and Departments are facing this challenge but it is not the case of the individual assessment of teachers in spite of its relevance. The assessment of teacher competence at university has been traditionally restricted to research, at least as an external evaluation linked to academic and economic recognition. However, such an important aspect as teacher competence is has been usually forgotten. Even though students’ evaluations of teaching are used in most universities, they don’t have any repercussion. In sum, assessing the quality of teaching requires, firstly, defining the construct of teacher competence, and on that basis, selecting indicators and variables aimed to obtain a valid and reliable measure of the construct. When defining the construct of teacher competence, main guidelines from the European Higher Education Space must be observed. Hence, one of our purposes is to incorporate these guidelines to the definition of the construct in order to get an appropriate means for the assessment of teacher competence in this new context. Innovations developed in this sense by a member of San Pablo- CEU team will be the basis for the construct validation. Next step after the definition of the construct will be the study of sources and instruments to collect the needed information to assess the evidence of teacher competence as previously defined. These pretensions become more important in these days when the law makes explicit that Autonomies can give teachers individual economic complements depending on their personal merits as teachers, researchers, and administrators. In short, this paper aims to clarify the assessment of university teachers’ competence as follows: 1. The definition of the construct of teacher competence in the context of the European Higher Education Space in our country. 2. A valid and reliable instrument for students’ evaluation of teaching through confirmatory factor analysis. Besides, multilevel analysis will inform on the effects of some variables (degree, subject, gender, percentage of pass…). These variables could influence on the assessment students make of their teachers in different hierarchical levels (student-subject-grade-faculty-university). 3. Finally, we discuss the necessity to elaborate a global model to assess the quality of teaching including some other instruments that will be suggested and also some guidelines to adjust all the available information. We present first results in the second year of a 3 years reserch proyect (Ref. SEJ2007-67064/EDUC)

Method

In this paper we focus on the definition of teacher competence in the context of the European Higher Education Space and on the analysis of the student survey looking for meeting that context. In this sense, we do a literature review in the new context. In other way, we make statistical analysis of a new student survey in this context, looking for construct validity. It includes descriptive analysis, exploratory factorial analysis, confirmatory factorial analysis and multilevel analysis. Our sample includes more than 12000 cases in a wide range of different schools (Law, Management, Humanities, etc.). We have been able to identify what items of the survey are working in a right or wrong way and it is possible to suggest some modifications.

Expected Outcomes

We try to tackle the changes that the context of the European Higher Education Space should involve in teaching assessment. We show the first results of the survey, and how our theoretician model of teaching competence fit to the data. The results of the Rasch model are presented as well, next to the problems located. We explain too the evolution, in a 4 year period, of the schools scores and the impact of the assessment. Finally, we point out some considerations to elaborate a global model to assess the quality of teaching including some other instruments that will be suggested and also some guidelines to adjust all the available information. That can contribute, as an example in Spain, to make future decisions about possible economical complements to teachers depending on their teaching merits.

References

ABALDE, E.; MUÑOZ, J. M. Y RÍOS DE DEUS, M. P. (2002). Evaluación Docente Vs. Evaluación De La Calidad. Ponencia del IIIer Congreso Virtual de AIDIPE. http://www.uv.es/aidipe/CongVirtual3/Ponencia3.htm#_ftn1 ACEVEDO, R. (2003). Factores que influyen en la competencia docente universitaria: un modelo jerárquico lineal. Tesis doctoral. Universidad Complutense. ALEAMORI, L. M. (1999). Student rating myths versus research facts fron 1924 to 1998. Journal of personnel evaluation in education. 13 (2), pp. 153-166. ÁLVAREZ ROJO, V., García E., Gil J., Romero S., Díaz E. (2002). Propuestas del Profesorado Bien Evaluado para Potenciar el Aprendizaje de los Estudiantes. Sevilla. Ice. 2000 BAREA, M.; GALÁN, A. et al. (2001). Análisis de las necesidades de formación del profesorado universitario y propuesta de mejora a partir de experiencias interuniversitarias y multidisciplinares. Proyecto de investigación subvencionado por el Ministerio de Educación, Cultura y Deporte (Programa de Estudios y Análisis). Inédito. Bernstein, D. (2008, January 1). Peer Review and Evaluation of the Intellectual Work of Teaching. Change: The Magazine of Higher Learning, 40(2), 48-51. (ERIC Document Reproduction Service No. EJ787531) Retrieved February 15, 2009, from ERIC database. Beyers, C. (2008, January 1). The Hermeneutics of Student Evaluations. College Teaching, 56(2), 102-106. (ERIC Document Reproduction Service No. EJ794374) Retrieved February 15, 2009, from ERIC database. BIGGS, J. (2006). Calidad del aprendizaje universitario (2ª Edición). Narcea, Madrid. Bornmann, L., Mittag, S., & Danie, H. (2006, December 1). Quality Assurance in Higher Education--Meta-Evaluation of Multi-Stage Evaluation Procedures in Germany. Higher Education: The International Journal of Higher Education and Educational Planning, 52(4), 687-709. (ERIC Document Reproduction Service No. EJ747399) Retrieved February 15, 2009, from ERIC database. Buckridge, M. (2008, June 1). Teaching Portfolios: Their Role in Teaching and Learning Policy. International Journal for Academic Development, 13(2), 117-127. (ERIC Document Reproduction Service No. EJ810001) Retrieved February 15, 2009, from ERIC database. BURCHELL, H. (1995). A useful role for competence statements in post-compulsory teacher education?. Assessment and evaluation in Higher Education, 20 (3), pp. 251-259. Burdsal, C., & Harrison, P. (2008, October 1). Further Evidence Supporting the Validity of Both a Multidimensional Profile and an Overall Evaluation of Teaching Effectiveness. Assessment & Evaluation in Higher Education, 33(5), 567-576. (ERIC Document Reproduction Service No. EJ809761) Retrieved February 15, 2009, from ERIC database. Costley, C. (2007, February). Work‐based learning: assessment and evaluation in higher education. Assessment & Evaluation in Higher Education, 32(1), 1-9. Retrieved November 10, 2008, doi:10.1080/02602930600848184 FERNÁNDEZ DÍAZ, M. J. y GALÁN, A. (1997). Desarrollo y situación actual de los estudios de eficacia escolar. Revista ELectrónica de Investigación y EValuación Educativa . 3 (1-3). GARCÍA RAMOS, J. M. (1997). Análisis factorial confirmatorio en la validación del constructo Competencia Docente del Profesor Universitario. Bordón, 49 (4), 361-391. GARCÍA RAMOS, J. M. y CONGOSTO, E. (2000): Evaluación y calidad del profesorado. En T. Glez Ramírez: Evaluación y gestión de la calidad educativa. Un enfoque metodológico. Málaga. Aljibe. Ghedin, E., & Aquario, D. (2008, November 1). Moving towards Multidimensional Evaluation of Teaching in Higher Education: A Study across Four Faculties. Higher Education: The International Journal of Higher Education and Educational Planning, 56(5), 583-597. (ERIC Document Reproduction Service No. EJ814151) Retrieved February 15, 2009, from ERIC database. GLASSICK C. E., M. T. HUBER Y MAEROFF G. I. (1997) Scholarship Assessed: Evaluation of the Professoriate. Jossey-Bass GLASSICK, C. E., HUBER, M. T., & MAEROFF, G. I. (1997). Scholarship assessed: Evaluation of the professoriate (An Ernest Boyer Project of the Carnegie Foundation for the Advancement of Teaching). San Francisco: Jossey-Bass. GONZÁLEZ SUCH, J. (2003). Modelos, procedimientos e instrumentos de evaluación de la actividad docente. Educación médica., jul.-sep. 2003, vol.6, no.3, p.20-21. GONZÁLEZ, J. Y PAGANI, R. (2003). Declaración de Bolonia: adaptación del sistema universitario español a sus directrices. En http://www.us.es/us/temasuniv/espacio-euro/adaptacion.html Kimball, S., White, B., Milanowski, A., & Borman, G. (2004). Examining the Relationship Between Teacher Evaluation and Student Assessment Results in Washoe County. PJE. Peabody Journal of Education, 79(4), 54-78. KNIGHT, P. T. (2006). El profesorado de educación superior. Formación para la excelencia (2ª Edición). Narcea, Madrid. Krull, E., Oras, K., & Sisask, S. (2007, October 1). Differences in Teachers' Comments on Classroom Events as Indicators of Their Professional Development. Teaching and Teacher Education: An International Journal of Research and Studies, 23(7), 1038-1050. (ERIC Document Reproduction Service No. EJ770299) Retrieved February 15, 2009, from ERIC database. Lombardi, J. (2008, January 1). To Portfolio or Not to Portfolio: Helpful or Hyped?. College Teaching, 56(1), 7-10. (ERIC Document Reproduction Service No. EJ787548) Retrieved February 15, 2009, from ERIC database. MARSH, H. W. y ROCHE, L. A. (2000). Effects fo grading leniency and low workload on students’ evaluation of teaching: popular myth, bias, validity or innocent bystanders? Journal of educational psychology, 92 (1), pp.202-228. McCracken, H. (2009, January 1). Best Practices in Supporting Persistence of Distant Education Students through Integrated Web-Based Systems. Journal of College Student Retention: Research, Theory & Practice, 10(1), 65-91. (ERIC Document Reproduction Service No. EJ796385) Retrieved February 15, 2009, from ERIC database. MCKEACHIE, W. J. (1997. )Studen Ratings: the validity of use. American Psychologist, 52 (11) pp. 1218-1225. MEDLEY, D.M., COKER, H. & SOAR, R.S. (1984). Measurement-based evaluation of teacher performance: An empirical approach. New York: Longman. MIGUEL DIAZ, M. de (2003). Evaluación y mejora de la actividad docente del profesorado universitario. Educación médica, jul.-sep. 2003, vol.6, no.3, p.22-25. Onwuegbuzie, A., Witcher, A., Collins, K., Filer, J., Wiedmaier, C., & Moore, C. (2007, January 1). Students' Perceptions of Characteristics of Effective College Teachers: A Validity Study of a Teaching Evaluation Form Using a Mixed-Methods Analysis. American Educational Research Journal, 44(1), 113-160. (ERIC Document Reproduction Service No. EJ782315) Retrieved February 15, 2009, from ERIC database. ORDEN, A. DE LA (1997). Desarrollo y validación de un modelo de calidad universitaria como base para su evaluación. Revista ELectrónica de Investigación y EValuación Educativa, 3 (1-2). Otto, J., Sanford, D., & Ross, D. (2008, August 1). Does Ratemyprofessor.com Really Rate My Professor?. Assessment & Evaluation in Higher Education, 33(4), 355-368. (ERIC Document Reproduction Service No. EJ803168) Retrieved February 15, 2009, from ERIC database. ROSADO M. J.; GALÁN, A.; VADILLO, O. et al. (2003). Sistema de evaluación del profesorado universitario para su contratación por las universidades de Madrid. Madrid, ACAP. SCHALOCK, M. D. (1993). Teacher productivity revised: Definition, theory, measurement and aplication. Journal of personnel evaluation in education, 8 (2), pp. 179-196. Smith, C. (2008, October). Building effectiveness in teaching through targeted evaluation and response: connecting evaluation to teaching improvement in higher education. Assessment & Evaluation in Higher Education, 33(5), 517-533. Retrieved November 4, 2008, doi:10.1080/02602930701698942 TEJEDOR, F. J. (2003). Evaluación de la docencia del profesorado: problemática y alternativas de mejora. En J. Cajide, M. A. Santos y A. Porto: Calidad educativa en contextos multiculturales. Universidad de Santiago de Compostela. TEJEDOR, F. J. (2003). Un modelo de evaluación del profesorado universitario. Revista de investigación educativa, 21, 1, 157-182. Tillema, H., & Smith, K. (2007, May 1). Portfolio Appraisal: In Search of Criteria. Teaching and Teacher Education: An International Journal of Research and Studies, 23(4), 442-456. (ERIC Document Reproduction Service No. EJ756892) Retrieved February 15, 2009, from ERIC database. TING, K. F. (2001). A multilevel perspective on student ratings of instrucion: Lessons form the Chinese Experience. Reserch in Higher Education. Vol 41, 5, pp. 637-653. van der Schaaf, M., & Stokking, K. (2008, June 1). Developing and Validating a Design for Teacher Portfolio Assessment. Assessment & Evaluation in Higher Education, 33(3), 245-262. (ERIC Document Reproduction Service No. EJ792198) Retrieved February 15, 2009, from ERIC database. VINKE, P. (1992). Multicriteria Decision-Aid, John Wiley and Sons White, F. (2008, September 1). Tenure and Promotion 101: What You Need to Know?. Online Submission, (ERIC Document Reproduction Service No. ED502681) Retrieved February 15, 2009, from ERIC database. YOUNG, S. y SHAW, D. G. (1996). Characteristics of Highly Effective Teachers: A Cluster Analysis. Paper presented at the annual meeting of the American Education Research Association, New York. ZABALZA, M. A. (2006). Competencias docentes del profesorado universitario. Calidad y desarrollo profesional (2ª edición). Narcea, Madrid. Declaración de Bolonia. Declaración conjunta de los Ministros Europeos de Educación. Bolonia, 19 de Junio de 1999 Declaración de Praga. Hacia el Área de la Educación Superior Europea. Declaración del encuentro de los Ministros Europeos en funciones de la Educación Superior en Praga, 19 de mayo del 2001. Comunicado de Berlín. “Construyendo el Espacio Europeo de Educación Superior” Comunicado de la Conferencia de Ministros con competencias en Educación Superior, celebrada en Berlín en 19 de septiembre del 2003. Comunicado de la conferencia de ministros europeos responsables de educación superior. Bergen. El Espacio Europeo de Educación Superior-Alcanzando las metas. 19-20 de Mayo de 2005. Reichert, S. y Tauch, C. (2005). Tendencias IV: universidades europeas. Puesta en práctica de Bolonia. Informe de la EUA. European University Association (EUA) (2005) Declaración de Glasgow: "Universidades fuertes para una Europa fuerte" ESIB – the National Unions of Students in Europe European Student Handbook on Equality and Equal Access EQUAL. "Integrated Guidelines for Growth and Employment" EQUAL. "Community Strategic Guidelines 2007-2013 - Cohesion policy in support of growth and jobs";

Author Information

Arturo Galán González

Universidad Nacional de Educación a Distancia (UNED)

Métodos de Investigación y Diagnóstico en Educación

Madrid

Isabel Muñoz

Universidad Pontificia Comillas

Educación, Médos de Investigación y Evaluación