Information Literacy (IL) is listed as XXI century key competence and a central dimension in all major digital and media literacy models, e.g., including DigComp 2.1 (Carretero, Vuorikari & Punie, 2017), and in many national school programs.

The low entry barriers to digital media allow practically everybody to produce and spread information across the globe from their computers or mobile phones. Traditional broadcast and news companies have lost their monopoly on information and our media environment evolved from a structured and well-defined landscape to a very cluttered and overcrowded space. In the so-called information society, the amount of information is so overwhelming that assessing the quality of information has become a pressing issue, as the Covid-19 pandemic crisis and the related infodemic have clearly illustrated (Zarocostas, 2020). Searching and evaluating information is a core life activity, be it for finding facts, facilitating decision-making or for pleasure and enjoyment (White, 2016), and it is fundamental for the political life of a community (Golding, 1994), as democracy is based on the assumption that citizens can make good choices based on reliable information. How someone searches, selects, evaluates and organises information has therefore been named one of the most critical skills for the XXI century (Ananiadou & Claro, 2009). According to some scholars, IL is the key to leverage on digital media to implement any effective life-long learning strategy (Kurbanoglu, 2012).

While IL models seem to converge on the key phases of information search, selection and management (e.g., as represented in the 7 Pillars model; SCONUL, 2011), teaching IL is not an easy task. Indeed, it entails both technical and procedural skills along with conceptual knowledge, and should be supported by sound research and an adequate understanding of what actually searching information means today.

Research on IL so far has been based on the (self-)assessment of IL skills (e.g., in the ICILS project; Fraillon et al., 2020), on measuring IL self-efficacy (Kurbanoglu, Akkoyunlu & Umay,  2006) and on capturing the online search behaviour with monitoring tools, such as URL timestamping (Gwizdka & Spence, 2006) or eye-tracking (Jiang, 2014), mostly using academic or job tasks as a benchmark. While all these approaches delivered and still deliver very useful results and insights, we believe that some recent developments changed the landscape and call for the development of new approaches:

1. Today, searching for information happens mainly through (personal) digital devices and the Internet, so that “searching” most commonly means “searching online” (this is indeed why IL is part of digital and media competence models).

2. The diffusion of digital devices and ubiquitous 24/7 Internet connectivity has made searching a capillary everyday task: people do not only search to do their homework or job tasks, but also to find recipes, transport schedules or health-related information.

3. Commonly used digital information search systems such as search engines have grown extremely complex and opaque to users, integrating AI algorithms and embracing personalization and making their use more complex and individualized.

We believe that new research methods are needed in order to collect data that can actually grasp the nature, variations and nuances of online information search practices today. Surveys, lab sessions with think-aloud protocols or eye tracking devices are definitely useful, but cannot pretend to observe anything similar to what happens when somebody is deciding what shoes to buy while laying on the sofa and eating a pizza.

In this paper, we present the collection of Search Stories as an innovative method to investigate IL practices (cf. Methods section), designed to preserve the ecology of data collection during  information search tasks in a complex and diversified environment.

The Late-teenagers Online Information Search (LOIS) project aimed to generate data from direct technologically-supported observation of real-life online search tasks. LOIS participants (target age group 16-20) are asked to install an ad hoc extension in their Firefox or Chrome browser, and are then asked to solve 4 information search tasks. While they navigate, the extension records: - Navigation Actions (NAs), i.e., time-stamped URLs of -- Search (S-NA): visits to search engines domains, search pages or Search Engine Results (SER) pages. -- Result (R-NA): visits to web pages that are not search engines. -The search queries they type or select from suggestions. -The ranking of the results they select when they click from a Search Engine Results (SER) page. -Any action of opening/closing tabs or new windows. After a data-cleaning phase, such information is compiled into a Search Story (SS), i.e., a representation of a task-related search session by an individual user. A SS is usually composed by different Search Episodes (SE), i.e., what happens between a S-NA and the next one; SEs are composed by multiple NAs (S-NA and R-NA). Metadata can be extracted from each SS, e.g., the overall number of actions, the number of new queries, the average time spent on SER pages, etc. This allows comparing different SSs and identifying similarities and differences. Visualizations are used in order to allow researchers to explore, compare and group SSs. Search stories can also be transformed into a narrative format, which makes them understandable to a broader audience. This can be used both to control accuracy with the participants through an ex-post control session, and to present the story for instructional purposes. Finally, as users have been profiled according to socio-demographic data and to Big 5 psychometric scales (McCrea & Costa, 1987), correlations can be calculated between the participants’ features and their search behaviour. The collection of Search Stories is also less time-consuming than organizing lab sessions, thus allowing the construction of larger datasets. Once a set of stories has been collected, the LOIS project plans to use data-mining algorithms to identify search patterns and classes of behaviors and to match them with task and participants profiles.

A first explorative study and a larger one on fake news collected over 150 SS from Swiss and Italian high-school and university students in 2020. The main LOIS data collection was conducted in spring 2021, collecting 604 SSs from 151 participants. The presentation at ECER will include the findings of all three studies. The analysis of SSs so far has demonstrated that, while IL can be conceptualized as a consistent construct, its practices vary, depending on searcher, task and situation. Some participants act differently on different tasks (e.g., on serious vs. leisure topics), generating short vs. long stories, or stories with longer stays on single pages vs. rapidly-paced ones. In the same way, SSs elicited differences in how participants handle the same task, even in cases where they actually achieve useful results and comparable satisfaction. Finally, the situation in which the search happens (time of the day; if done in a unique stretch or in separate moments; etc.) seems to influence the search behaviour. Observed through the lenses of SSs, online information search resembles more a subtle art than an exact science, guided by principles more than rules. This might hint to develop a teaching approach more responsive to the nuances of personal needs and situations. The SS approach is still under development, and surely meets some limitations. While favoring an ecological data collection, it should consider technical issues when it comes to data completeness. Indeed, its technical tools should be adapted on an ongoing basis to keep up with search engines and browsers developments, and a mobile version of the extension is not yet available. Nonetheless, we believe that this approach is promising for a better understanding of IL, and to support more effective IL education. The LOIS browser extension and all collected data will be released open source.

Ananiadou, K., and Claro, M. (2009). 21st Century Skills and Competences for New Millennium Learners in OECD Countries. OECD Education Working Papers, 41, OECD Publishing. DOI: 10.1787/218525261154 Carretero, S., Vuorikari, R. and Punie, Y. (2017). DigComp 2.1: The Digital Competence Framework for Citizens with eight proficiency levels and examples of use, EUR 28558 EN, DOI:10.2760/38842 Fraillon, J., Ainley, J., Schulz, W., Friedman, T., & Duckworth, D. (2020). Preparing for Life in a Digital World. IEA International Computer and Information Literacy Study 2018 International Report. Springer Open. Golding, P. (1994). Telling stories: Sociology, Journalism and the Informed Citizen. European Journal of Communication, 9(4), 461-484. Gwizdka, J., & Spence, I. (2006). What can searching behavior tell us about the difficulty of information tasks. A study of Web navigation. Proceedings of the American Society for Information Science and Technology, 43(1), 1-22. Jiang, T. (2014). A clickstream data analysis of users' information seeking modes in social tagging systems. In Proceedings of the 9th iConference, 314–328. Kurbanoglu, S. S., Akkoyunlu, B., & Umay, A. (2006). Developing the information literacy self‐efficacy scale. Journal of documentation, 62(6), 730-743. Kurbanoglu, S. (2012). An analysis of the concept of information literacy. Proceedings of the International Conference of the Media and Information Literacy for Knowledge Society (June 24-28), 1-42. McCrae, R. R., & Costa, P. T. (1987). Validation of the five-factor model of personality across instruments and observers. Journal of personality and social psychology, 52(1), 81-90. SCONUL (1999). Information Skills in Higher Education: A SCONUL Position Paper. London: Society of College, National and University Libraries. White, R. W. (2016). Interactions with Search Systems. Cambridge University Press. Zarocostas, J. (2020). How to fight an infodemic. The Lancet, 395(10225), 676.