Session Information
12 SES 06 A JS, Joint Symposium - From Log Data to Research Data: Quality, Documentation, Reuse, and Protection in Large-Scale Educational Studies - NW 09, NW 12 and NW 27
Joint Symposium Session
Contribution
Log data (i.e., event data documenting individuals’ interactions with a digital environment) have become an important source of information in empirical educational research (Becker et al., 2022; Goldhammer et al., 2021). They provide fine-grained insights into learners’ behavior and performance-related processes, thus extending beyond what can be captured with traditional questionnaire or test score data. However, log data also pose specific challenges for data sharing and reuse. Their high volume, technical nature, and strong context dependence make it difficult to implement the FAIR principles (Wilkinson et al., 2016), in particular with respect to reusability. Consequently, log data remain underutilized in secondary analyses despite their high scientific value and reuse potential (Becker et al., 2022). One of the central challenges in making log data reusable is the provision of high-quality documentation. Log data typically cannot be interpreted without extensive background knowledge, much of which is held implicitly by the primary researchers. This concerns not only the structure of the log data itself but also the digital system that generated them. The meaning of a log event, such as a mouse click, can only be understood in relation to the technical elements of the system, including which interface elements can be interacted with and what is displayed on the screen at the time of the interaction. Codebooks and purely textual descriptions are therefore often insufficient to convey this information. In addition, log data typically undergo a multi-step processing procedure to derive interpretable indicators of learners’ behavior and performance-related processes (Goldhammer et al., 2021). This procedure includes data preprocessing, the extraction of low-level features, and the modeling of process indicators (Kroehne & Goldhammer, 2025). Without transparent documentation of these steps, the interpretability, reproducibility, and scientific reuse of the data are severely limited. Taken together, these challenges highlight the need to develop high-quality log data documentation strategies. Our presentation highlights these challenges and illustrates practical approaches to addressing them. Our examples draw on documentation strategies from large-scale assessments such as PIAAC (OECD, 2013), PISA (OECD, 2016), and TIMSS (Martin et al., 2020), as well as our own experience documenting log data from digital learning environments. For instance, we compare different methods for documenting the relationship between log data and the underlying digital environment, ranging from static documentation formats to log data replays, and discuss their respective strengths and limitations. The presentation concludes with concrete recommendations and proposed minimum requirements for documenting log data.
References
Becker, B., Neuendorf, C., & Jansen, M. (2022). Nutzung von Logdaten in der empirischen Bildungsforschung—Eine Bedarfsanalyse. https://doi.org/10.5281/ZENODO.7030996 Goldhammer, F., Hahnel, C., Kroehne, U., & Zehner, F. (2021). From byproduct to design factor: On validating the interpretation of process indicators based on log data. Large-Scale Assessments in Education, 9(1), 20. https://doi.org/10.1186/s40536-021-00113-5 Kroehne, U., & Goldhammer, F. (2025). Software Tools for Analyzing Log Data. In L. Khorramdel, M. Von Davier, & K. Yamamoto (Eds.), Innovative Digital-Based International Large-Scale Assessments (pp. 657–691). Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-90951-1_27 Martin, M. O., von Davier, M. & Mullis, I. V. (2020). Methods and Procedures: TIMSS 2019 Technical Report. TIMSS & PIRLS International Study Center. OECD. (2013). Technical Report of the Survey of Adult Skills PIAAC (Second Edition). OECD Publishing. OECD. (2016). PISA 2015 assessment and analytical framework. OECD Publishing. https://doi.org/10.1787/19963777 Wilkinson, M. D., Dumontier, M., Aalbersberg, Ij. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., da Silva Santos, L. B., Bourne, P. E., Bouwman, J., Brookes, A. J., Clark, T., Crosas, M., Dillo, I., Dumon, O., Edmunds, S., Evelo, C. T., Finkers, R., … Mons, B. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data, 3(1), Article 1. https://doi.org/10.1038/sdata.2016.18
Update Modus of this Database
The current conference programme can be browsed in the conference management system (conftool) and, closer to the conference, in the conference app.
This database will be updated with the conference data after ECER.
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance, please use the conference app, which will be issued some weeks before the conference and the conference agenda provided in conftool.
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.