The negative diagnosis of the Chilean educational system in terms of its general quality and its social inequity led, in the 2000, to the implementation of changes directed at public school teachers’ performance. Although most teachers had obtained income increments since 1990, these improvements remained dissociated from the quality of their teaching performance. Thus in 2002 the Ministry of Education decided to implement two systems of evaluation intended to establish a more evident connection between teacher performance and teacher salaries. This Ministry created the National Teacher Evaluation System, aimed at appraising the performance of all public school teachers in the country in a mandatory way. In addition, the same Ministry created the Certification of Teaching Excellence program (CTE), the focus of this paper, which was aimed at detecting and recognizing high performance among teachers in the public system.

The CTE started in 2002 and the evaluation is based on national standards for teaching quality (Mineduc, 2001). The CTE offers more than 35 certifications in different subject areas for pre-school, primary and secondary teachers. Participation is voluntary (some 2000 teachers provide the evidence for evaluation each year) and about 1/3 of applicants receive the certification, with which they gain access to a monetary incentive of one additional month of salary per year for ten years.

The program theory reconstructed in our study reveals that, at the level of the educational system, the program is expected to increase teacher salaries based on individual performance, retain good teachers in the classroom, and foster peer mentoring and collaboration. At school level it is expected that certified teachers encourage their peers to participate in the program. Finally, at individual level certified teachers are supposed to benefit from social recognition and to increase their professional self-esteem.

In many ways the CTE shares various objectives with certification processes implemented in the United States (Cavalluzzo, 2004; Ellett & Teddlie, 2003; NRC, 2008). Nonetheless, in order to be able to meet these challenging expectations, the CTE program must show evidence that its certification decisions are valid and reliable, and thereby, fair to involved stakeholders. Manzi et al. (2007) reported evidence supporting CTE’s construct validity. However, the consequential validity of this program has not been addressed. About that, Messick (1994, 1998), Linn (1997) and others have argued in favor of the inclusion of observed consequences in the validation of tests and other assessments (Shepard, 1997; Lane, Park & Stone, 1998; Moss, 1994, 1998, AERA, APA & NCME, 1999; JCSEE, 1988).

The study reported here aims at contributing to the validity evidence regarding CTE, specifically its consequential validity. The objectives are:

1.       Identify expected consequences of CTE, according to the perception of program designers (developing a program theory).

2.       Identify observed consequences (expected and unexpected), as perceived by certified and unsuccessful applicants and school leaders.

Particular research questions are: Which are the intended consequences of CTE in view of program designers? What consequences of CTE are perceived by certified teachers, unsuccessful candidates and principals?  Are they unexpected or intended consequences?

AERA, APA & NCME (1999). Standards for educational and psychological testing. Washington, DC: APA Hsieh, H. and Shannon, S. (2005) Three Approaches to Qualitative Content Analysis. Qualitative Health Research 2005; 15; 1277. JCSEE (1988). The Personnel Evaluation Standards. Newbury Park, CA: Corwin Press. Lane, S. y Stone, C.A. (2002). Strategies for examining the consequences of assessment and accountability programs. Educational Measurement: Issues and Practice. Nº21(1), pp. 23-30. Lane, S., Park, C.S., y Stone, C.A. (1998). A framework for evaluating the consequences of assessment programs. Educational Measurement: Issues and Practice. Nº17(2), pp. 24-28. Linn, R. L. (1997). Evaluating the validity of assessments: the consequences of use. Educational Measurement: Issues and Practice. Nº16, pp. 14-16. Manzi, J., Araya, C., González, R., Barros, E., Bravo, D., Peirano, C., Cadiz, J., y Torres, D. (2007). Validity evidence for the certification of teaching excellence in Chile. Chicago: Document presented in AERA Anual Meeting (March, 13). Mehrens, W. (1997). The consequences of consequential validity. Educational Measurement: Issues and Practice. Nº16, pp. 16-19. Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher. Nº23(2), pp. 13-23. Messick, S. (1998). Test validity: a matter of consequence. Social Indicators Research. Nº45, pp. 35-44. Mineduc (2001). Marco para la Buena Enseñanza. Documento publicado por el Ministerio de Educación - Chile. Moss, P.A. (1992). Shifting conceptions of validity in educational measurement: implications for performance assessment. Review of Educational Research. Nº62. pp. 229-258. Moss, P.A. (1998). The role of consequences in validity theory. Educational Measurement: Issues and Practice. Nº17(2), pp. 6-12. NRC - National Research Council (2008). Assessing accomplished teaching. Advanced – level certification programs. Washington, D.C.:The National Academic Press. Shepard, L.A. (1997). Evaluating centrality of test use and consequences of test validity. Educational Measurement: Issues and Practice. Nº6, pp. 71-86.