In PISA, as in all cross sectional large scale assessments, associating background variables with performance differences can only be done with great caution. For example, in Germany we find a performance difference of 73 points on the PISA scale between students with a migration background and native students. We also find a performance difference of 73 points between students who have more than 4 hours and students with less than 2 hours of science instruction per week. What happens if we control the “migration” effect for social background, for instruction time or for the language spoken at home? Which variables need to be considered? Based on this kind of example this paper addresses the question how performance differences are related to background variables. Having more than one level of analysis in a study like PISA, namely the country, the school, the class and the individual level, there are countless ways of arbitrarily defining relationships between different levels and variables. A large array of analyses is carried out in each PISA cycle. However, only a small number of the results gets public attention, e.g. the association of performance with students’ migration background, with their socio-economic background or with certain school variables, whereas other interesting and instructive associations find hardly any attention. Therefore the PISA results are at risk of being selective, interpreted causally, not sufficiently controlled or used to confirm popular stereotypes. The paper shows a selection of results from PISA 2003 and 2006 and emphasizes that the scope of our results and the variables which we control for have to be thoroughly considered.