There is a three-fold universal challenge in education when a promising program (one that has a limited amount of evidence of its effectiveness) is introduced and implemented in districts, schools and classrooms. First, even before a program is implemented in schools, the developer must be able to articulate the program logic model – describing the critical supports of implementation (e.g., professional development (PD), administrative support, daily schedule, etc.), the changes expected to occur in the program once implementation supports are delivered (e.g., teachers’ pedagogical knowledge increases after PD), and ultimately improvements in student experiences and/or social, emotional, physical, and academic development depending on the program focus. Second, the defined program model must be fixed enough to enable assessment of the degree to which critical elements of the program were supported and eventually implemented (i.e., fidelity of implementation). The model, however, must be flexible enough to accommodate modifications when and if necessary to better align with the unique characteristics and needs of any new environment and varying political landscapes. The justification of any program modification must be based on data collected as a part of program monitoring, performance improvement, feedback from program deliverers, or another system that answers questions concerning the degree of implementation, the quality of delivery, and the perceived usefulness for both deliverers (e.g., teachers) and recipients (e.g., students) of the program.  Finally, an assessment must be made as to the degree to which modifications are more minor in nature or are substantial enough to change the fundamental structure and focus of the program.

This symposium presents three papers on the topic of program modeling, implementation, adaptation, and modification.  The first paper presents an overview of fidelity of implementation and walks through a system of measurement developed for a large initiative funded by the US Department of Education.  The following two papers present case studies of curricula and educational approaches and their individual experiences with the development of the program logic model, initial rounds of implementation, and subsequent modifications to the program.  Specifically, the second paper presents data about a national evaluation of an expanded learning time (ELT) model in the United States, and how initial qualitative implementation data shaped the program’s evolution and modification to the model. In subsequent years, these data helped inform the development of a fidelity index used to more systematically collect data and assess schools on their levels of implementation.  The third and final paper focuses on issues surrounding two related research projects that examine the implementation of the new framework curricula in Hungary. In particular, this presentation addresses how and to what extent the new framework curricula were built into the pedagogical programs and local curricula of different schools from grades 1 to 12.

Our proposed discussant, an expert on the system of education in Iceland, will provide perspective on the universal issues of program modeling, implementation, and adaptation presented as examples in the US (first and second papers), Hungary (third paper), and Iceland. As fidelity of program implementation is increasingly recognized as an essential component of education research regardless of context, outlining systematic measurement of program modeling is becoming a universal necessity.

N/A