The increasing availability of various data sources and the repeated call to use data to improve instruction is described as a part of an expanding datafication in education. It is assumed that data will become increasingly important, e.g., in education policy, but also in teachers' decision-making (Jarke & Breiter, 2019). Educational research shows that data-informed decision making plays a key role for improvement of instruction and school improvement (Lai & Schildkamp, 2013). Therefore, data use is promoted through education policy. In most European countries different data sources like school self-evaluations or student achievement data are made available to teachers to improve instruction. Creemers & Kyriakides (2008) point out the need to collect multiple data on instruction and student achievement. In Germany teachers can use routinely collected student achievement data from state-wide comparison tests and centralized exit examinations and teachers are encouraged to conduct regular self-evaluations that can provide a range of different information (Thiel et al., 2019). In Germany, state-wide comparison tests and self-evaluations are primarily designed to improve instruction, while centralized exit examinations fulfill the primary function of certifying student achievement. Therefore, state-wide comparison tests provide more comprehensive information then centralized exit examinations. When conducting self-evaluations, teachers themselves decide on the subject they want to evaluate. These different goals, and thus different levels of information available, can lead to differences in the use of these data sources (Visscher & Coe, 2003). Another problem resulting from the diversity of data sources and the challenge of deriving adequate improvement measures from the data is discussed as the Data Rich Information Poor Syndrome, first introduced in medicine, and then adapted in the field of education (Goodwin, 1996; Slotnik & Orland, 2010). Data itself is not sufficient to initiate improvement measures, and too much data can cause an “overload” that prevents appropriate data use. On the other hand, multiple data provide a broader and more reliable information base and prevent a possible narrowing of data informed decisions by the specific focus of single data sources (Koretz, 2003). The integration of different complementary data sources might have a positive impact on data informed decision making, for example, if learning outcomes are analysed in consideration of the corresponding instruction processes. Most studies investigated the use of single data sources or do not separate between them. Little is known whether multiple data sources are used to improve instruction. In light of the assumed datafication of education the research questions are: What data sources do teachers use to improve instruction? What data are teachers using simultaneously to improve instruction?

The study analyses teachers’ use of self-evaluation, state-wide comparison tests and centralized examinations. It is based on the German large scale national assessment study 2012, which was conducted by the Institute for Educational Quality Improvement (Pant et al., 2013). The following analyses are based on N = 3028 questionnaires from mathematics and science teachers in N = 1171 secondary schools who have experience with at least one of the three data sources. Of these, 60% have experience with state-wide comparison tests, 46% with centralized examinations, and 59% with self-evaluation. In the sample, 43% are upper secondary teachers, 56% are female, the average age is 48.7 years (SD=9.4), and the average professional experience is 21.3 years (SD=12). Descriptive analyses and Regularized Partial Correlation Networks (Epskamp & Fried, 2018) were used as a method to identify measures based on the use of multiple data. Partial Correlation Networks provide coefficients that control for all correlations with other variables in the correlation matrix. This is necessary to identify improvement measures based on more than one data source. Other advantages of this method are an easy-to-interpret graphical overview of the large number of coefficients and consideration of the statistical problems of multiple testing.

A significant number of teachers use data from state-wide comparison tests (70%), centralized exit examinations (75%) and self-evaluations (80%) for at least one improvement measure. Teachers use the data to introduce new teaching methods, develop measures/interventions for individual student support, as a reason to participate in professional development, to refine the school's curriculum and to improve communication within the teaching staff. However, slightly more than half of the respondents indicated rarely or not at all implementing measures when evaluating each improvement measure separately (12 to 30 %). There are no significant differences in the improvement measures between the three data sources, except for curriculum refinement, which was done more often based on self-evaluation data. The results also show simultaneous use of multiple data for improving instruction. For example, significant correlations are shown between the improvement measure introduction of new teaching methods as well as work on the curriculum and in-service training are shown. This applies in particular to student performance data (from state-wide comparison tests, centralized exit examinations). Simultaneous use in combination with self-evaluation data is less frequent. In summary, on the one hand the results point to the potential of a joint analysis of different data sources, on the other hand it is shown that not all theoretically assumed advantages are utilised. Another finding – important in terms of government policy–is that centralized exit examinations data is used to improve teaching, although this data source has a different intention (selection of students). The results indicate that further research is needed on: the design of data itself and data systems to improve the use of multiple data, the alignment of data, educational goals and teacher training; as well as in-depth analyses of decision-making processes.

Creemers, B. P. M., & Kyriakides, L. (2008). The Dynamics of Educational Effectiveness: A Contribution to Policy, Practice and Theory in Contemporary Schools. London: Routledge. Epskamp, S., & Fried, E. I. (2018). A tutorial on regularized partial correlation networks. Psychological Methods, 23(4), 617–634. Goodwin, S. (1996). Data rich, information poor (DRIP) syndrome: Is there a treatment? Radiology Management, 18(3), 45–49. Jarke, J., & Breiter, A. (2019). Editorial: The datafication of education. Learning, Media and Technology, 44(1), 1–6. https://doi.org/10.1080/17439884.2019.1573833 Koretz, D. (2003). Using Multiple Measures to Address Perverse Incentives and Score Inflation. Educational Measurement: Issues and Practice, 22(2), 18–26. Lai, M. K., & Schildkamp, K. (2013). Data-based Decision Making: An Overview. In K. Schildkamp, M. K. Lai & L. Earl (Hrsg.), Data-based Decision Making in Education. Challenges and Opportunities (S. 9-21). Heidelberg, London, New York: Springer. Pant, H. A., Stanat, P., Schroeders, U., Roppelt, A., Siegle, T., & Pöhlmann, C. (2013). The IQB National Assessment Study 2012. Berlin: Institute for Educational Quality Improvement. Slotnik, W.J & Orland, M. (2013). Data Rich but Information Poor. Education Week, March 27, 2013. Thiel, F., Tarkian, J., Lankes, E.-M., Maritzen, N., Riecke-Baulecke, T., & Kroupa, A. (Hrsg.). (2019). Datenbasierte Qualitätssicherung und -entwicklung in Schulen: Eine Bestandsaufnahme in den Ländern der Bundesrepublik Deutschland. Springer Fachmedien Wiesbaden. Visscher, A. J., & Coe, R. (2003). School Performance Feedback Systems: Conceptualisation, Analysis, and Reflection. School Effectiveness and School Improvement, 14(3), 321-349.