This methodological paper explores the application of network ethnography—a novel and ‘developing’ approach put forth by Stephen Ball (2016)—within a larger empirical research project on the ‘ILSA industry’, a study of contractors involved in international large-scale assessments and, more broadly, of education privatisation. 

In the last two decades, education policy, practice, and research have been transformed by the appearance and growing influence of international large-scale assessments (ILSAs). Alongside growing participation in ILSAs, the global ILSA industry is also expanding and becoming more complex. While scholars have extensively studied the impact of ILSAs on education practice and policy, they have largely ignored the contractors involved in developing ILSAs: mainly private companies, including for-profit and not-for-profit; research institutes; and universities. 

During the first part of this project, we used network ethnography techniques to map the global ILSA industry by identifying the actors involved in the development of ILSAs, their roles, and their network relationships. Here, our particular aim is to analyse ways in which we, as scholars, ‘become with our methodology’ (Law 2004) and come to terms with the ‘messiness’ of research (Addey and Piattoeva 2022). While most methodologies in the social sciences are presented as standardised procedures that, when applied, lead scholars to the same findings and conclusions, we use our application of network ethnography as a case study of the complex, subjective, and deeply personal process of research. We explore how our methodological choices are influenced by the nature of the research field, our access to it, and our perception of it—and how these choices shape us, our research process, and the knowledge we produce. In doing so, we propose that ‘heterogeneity and variation’ (Law 2004) are an inherent part of any methodological application.

Drawing on Science and Technology Studies (STS) to understand the constitutive role of methodology and the performativity of knowledge-making, intervening in the world but also in the researcher (Law & Singleton 2013, Rimpiläinen 2015), we ask questions such as: How does our encounter with the ILSA contractors and space generate practices of method that shape the way we make knowledge? How are we deeply implicated in our epistemological practices and the worlds in which we are intervening? This approach shows how our methodology is performative: as we take decisions about our practices of method, it constructs what we are studying and ourselves. With an STS approach to our application of network ethnography, we present the complex and provisional nature of knowledge.

Finally, we discuss the challenges of visually representing the network of ILSA contractors. We apply Galloway’s (2011) notion of ‘conversion rules’ to make explicit the categories and relationships which give structure to this network, as well as accounting for absence and what went unrepresented in our attempt. We conclude by exploring what value might be drawn from this set of visualisations and, looking forward, what new approaches might be inspired by its limitations.

Methodologically, the ILSAINC project was inspired by Ball’s use of network ethnography to study mobilities and interactions that move and fix policy across transnational and intra-national spaces. This methodological approach lends itself to the study of contracted expertise in ILSAs—one aspect of the global trend of education privatisation—which is concerned with which actors enter this business space, how they use this space, and how education privatisation is being transformed by the role that the private sector plays in international testing. Described by Ball and Junemann (2012) as a joining of social network analysis with qualitative methods, network ethnography—more than social network analysis alone—aims to ‘to capture detail on incommensurate yet meaningful relationships’ (Ball and Junemann 2012, citing Howard 2016, p. 550). To do this, Ball describes mapping, following, visiting, and questioning actors, lives, stories, conflicts, money, and things, in order to understand how policy travels, who is involved in the moving and fixing across spaces, and how spaces are reconfigured as a consequence. In carrying out this network ethnography, this project drew on document analysis and in-depth interviews in order to understand the actors, relationships, and stories which constitute the dynamic network of contracted expertise in ILSAs. This paper, however, is interested in the relationship between methodological choices and knowledge production more broadly. STS asks us to analyse how we ‘become with our methodology’ and how our choices shape our research: our decisions construct what we are studying, and ourselves. Limited by time and capacity, we took decisions about which sources of data would be included and which would—or could—not. We decided how data would be organised, categorised, and labelled. On an ethical level, we considered where to draw lines of privacy and confidentiality, and where to set boundaries between the personal and the professional, the public and the private. As we mapped the ILSA network and constructed a visual representation, we took choices—some general, some specific to the nature of this network; some deliberate, some by necessity—that shaped the knowledge we produced. This process demonstrates how the methodology of network ethnography is, rather, ‘methodologies’: in each case, shaped by the nature of the space and by the choices of the researcher, and in turn shaping the knowledge produced and the researchers themselves.

This exploration of our own methodology speaks not only to network ethnographers but to qualitative researchers more broadly. It is a call for both new and established methodological approaches to be examined in action, as they are applied, as a way of challenging the common presentation of research methods as standardised procedures leading to replicable findings and conclusions. Rather, an examination of ‘methodology in action’ reveals the ‘messiness’ of research and the researcher’s role in constructing what is analysed. We pose that the application of any methodology requires the researcher to adapt their approaches and techniques to the unique natures of the field of study, where choices taken throughout the research process shape the knowledge produced. Moreover, this process shaped our own feelings and perceptions of ourselves, through ethical decisions we took regarding the data we collected. We thus also call attention to the impact that research has on the researcher as well as the research subjects and the production of knowledge. Ultimately, we propose that ‘heterogeneity and variation’ (Law 2004) are an inherent part of any methodological application.

Addey, Camilla and Nelli Piattoeva (Eds). 2022. Intimate accounts of education policy research: The practice of methods. Oxon: Routledge. Ball, S. 2012. Global Education Inc.: New Policy Networks and the Neoliberal Imaginary. Oxon: Routledge. Ball, S. 2016. “Following Policy: Networks, Network Ethnography and Education Policy Mobilities.” Journal of Education Policy 31 (5): 549–566. Ball, S. and C. Junemann. 2012. Networks, New Governance and Education. Policy Press. Galloway, A. 2011. “Are Some Things Unrepresentable?” Theory, Culture & Society 28(7-8): 85–102. Howard, P. N. 2002. “Network Ethnography and the Hypermedia Organization: New media, new organizations, new methods.” New Media and Society 4: 550–574. Junemann, C., S. J. Ball, and D. Santori. 2015. “Joined-up Policy: Network Connectivity and Global Education Policy.” In Handbook of Global Policy and Policy-Making in Education, edited by K. Mundy, A. Green, R. Lingard, and T. Verger. Hoboken, NJ: Blackwell. Law, J. 2004. After Method: Mess in Social Science Research. London: Routledge. Law, J., and V. Singleton. 2013. “ANT and Politics: Working in and on the world.” Qualitative Sociology 36 (4): 485–502. Rimpiläinen, S. 2015. “Multiple Enactments of Method, Divergent Hinterlands and Production of Multiple Realities in Educational Research.” International Journal of Qualitative Studies in Education 28(2): 137-150.