As the OECD envisages measuring all countries along a single metric of learning outcomes, the main testing agencies recognise that in order for data to have national policy value, testing instruments and collection methods need to value context specificity and be sensitive to diversity. But how do the centres of calculation face the challenge of standardising diversity? This paper explores how testing agencies are challenged by the need to ensure that the data are both context-specific but also validly comparable across different socio-economic and cultural contexts. I look into UNESCO’s Literacy Assessment and Monitoring Programme (LAMP) and PISA for Development which both aim to provide country-specific international data for countries with greater socio-economic and cultural differences (including European countries). Applying Actor-Network Theory to understand the encounter of standardisation and hybrids, I explore how standardized testing instruments in international assessment programmes value, accommodate and black box diversity. I questions why and how the monster of diversity is first loved and then killed (Latour 2011). I argue that although diversity is valued to produce nationally relevant policy knowledge, the priority to generate internationally comparative indicators and comparative knowledge is a strong incentive to accommodate diversity through the standardization of materials and procedures.