Implicit, or unconscious, biases operate outside of one’s conscious mind, and contribute to discriminatory behaviours and micro-aggressions (Devine, 1989). These biases are influenced by lived experiences, media, and societal stereotypes (Schwarz, 2000). A lack of awareness of implicit biases contributes to their persistence; therefore, unlike explicit biases, they cannot be amended by learning (Devine 1989; Devine, Forscher, Austin, & Fox, 2012). In fact, implicit biases are frequently found to exist when explicit measures (e.g., measured using self-report) indicate high levels of multicultural competency, anti-racism, or anti-ableism (Devine et al., 2012; Levine, Park, & Kuo, 2020).

Gender bias is an ever-present component of educational institutions, contributing to pervasive inequities in gender, ability status, race, and socioeconomic statuses. Hegemonic norms consistently hold young women back from fulfilling their potential, especially in Science, Technology, Education, and Mathematics, or STEM fields (Fawcett Society, 2017). The Junior Cycle (lower second level education) Science curriculum may not be compulsory in Ireland, however it is studied by approximately 90% of students; the majority of whom are male. However, just 1 in 6 students study Physics for the Senior Cycle, and regrettably, the ratio of male to female students grows to become greater than 3:1 by that level (The STEM Education Review Group, 2016). In England and Wales, science is a compulsory subject in which female students outperform male students, and yet they do not continue to study physics (Macdonald, 2014). Female students have reported that they feel science is for male students and when they find science challenging, they attribute it to their gender (Cornish, 2016). Concerns about the disparate outcomes for female students relative to male students in Physics has led to the Institute of Physics (IOP) recommending that unconscious biases be addressed  (IOP, 2014).

School textbooks contribute to the development of school culture and therefore cannot be ignored when looking at their implicit impression on students (Martins and Garcia, 2016). Yet, there is limited recently published literature examining gender bias in textbooks. Given the evidence of a male dominance within science textbooks (Sunar, 2011; Kahveci, 2010) and the lack of a gender bias measurement tool, Parkin and Mackenzie (2017) developed the Gender Bias 14 (GB14) to assess gender in textbooks. GB14 examines images and illustrations, statuses of person(s) represented, and the use of language (e.g., pronouns and gendered words; Rose, Spinks and Canhoto, 2015).

Parkin and Mackenzie used the GB14 tool to analyse the content of a Key Stage 3, 2014 National Curriculum science textbook in the UK and found that 16 of 18 chapters in that book were highly male biased (more male images, more male role-models, more male pronouns, more male-gendered words). The current study sought to replicate the study conducted by Parkin and Mackenzie using a qualitative approach and intercoder reliability processes. During the initial process of reviewing the codebook, the authors identified items in the codebook that were unable to be scored consistently across the review team, and other items that were deemed to be beyond the scope of the study (e.g., analysing discourse for genderedness). As such, a new codebook was developed by the team for the purpose of advancing Parkin and Mackenzie’s original work. This paper will present a revised Gender Bias Tool and will describe its development and validation process conducted by an interdisciplinary research team. The advancement of the Gender Bias Tool has significant implications for all areas within the STEM fields, given the importance of representation for ameliorating disparate outcomes for females in these fields, drawing attention to the development of materials to be consumed by students and their potential impact on outcomes for those students.

The two Principal Investigators of the project identify as male and female, with differing disciplinary backgrounds (physics and counseling/disability studies) and nationality/cultural background (Irish and American), respectively. The coding team consisted of the first author/PI and two graduate students with advanced qualitative methods training. The second author/PI oversaw the intercoder reliability (ICR) process, including managing the data and calculating kappa’s throughout the process. The textbook comprises of 38 modules, and four modules, or 10.5% of the content, were coded in order to establish ICR, which is within the typical range of the proportion of data multiply coded (O’Connor and Joffe, 2020). The adaptation of the codebook and ICR took place in two rounds to reach satisfactory reliability; a round of independent coding was followed by a meeting to discuss the misses and to arrive at a consensus agreement on codebook modifications needed to satisfy differences that arose (Campbell et al., 2013). Kappas for the modules that were coded to establish ICR have been calculated as ranging from 0.67 to 1.0, indicating substantial to almost perfect agreement (Landis & Koch, 1977). Following the first round of coding, two items were dropped that attempted at assigning gender identification (male/female) to individual words. Despite the fact that the original codebook contained an extensive list of examples, the authors decided that certain interpretations were too subjective and based on cultural stereotypes that would render broad agreement very difficult. In addition, replicating the coding of these words would be implausible for future research teams given the varied nature of language used in textbooks. Two additional items were removed from the codebook following the second round of coding that addressed pronouns for similar reasons. The resulting codebook consists of 14 items and corresponding instructions for coding all images, photographs, illustrations, diagrams, and charts. This paper will utilize a qualitative case study approach to presenting the authors’ experiences of adapting the codebook as an interdisciplinary team, including introducing the team members' positionalities (e.g., hard sciences, social science, qualitative vs quantitative, our genders, races, nationalities, etc.), and explaining our frames of reference for conceptualizing and studying gender based on our scientific orientations. This will be followed by a detailed description of the inter-coder reliability process.

The importance of representation for underrepresented groups and minorities in STEM cannot be understated. To date, minimal empirical attention has been paid to the development of classroom materials and textbooks for STEM classrooms. The Revised GB14 tool allows educators and researchers to assess the representations of gender in the materials that are being considered for distribution to students. Further, it is hypothesized that there will be an overwhelmingly male representation prevalent in most materials. As such, increasing educators’ awareness of the importance of representation in class materials is a crucial step in advancing equity for underrepresented groups in STEM fields. Not only is gender equity important for the diversification of the STEM fields, it is imperative that researchers and educators recognize that this issue contributes to persisting societal inequity. Graduates with degrees in STEM are consistently projected as having higher starting salaries relative to other degree areas, furthering the gender wage gap in America and the European Union (Center for American Progress, 2020; European Commission, 2020; National Association of Colleges and Employers, 2019). Materials such as textbooks that are distributed to students at impressionable stages of development should be representative of the desired state of STEM: inclusive, equitable and open to all who are interested in its pursuit. The Revised GB-14 (RGB-14) tool has significant implications for STEM. In addition to the ability to replicate and evaluate the genderedness of a variety of materials (e.g., textbooks, handouts, websites, and other mediums that represent information to students), the RGB-14 creates a starting point for its evolution. Future iterations of the RGB-14 could determine the representation of race and ability status represented in materials distributed to students. The dearth of representation of marginalized groups in STEM must be addressed at all levels of education, and in all materials being distributed to students.

Campbell, J. L., Quincy, C., Osserman, J., & Pedersen, O. K. (2013). Coding in-depth semistructured interviews: Problems of unitization and intercoder reliability and agreement. Sociological Methods & Research, 42: 294–320. https://doi.org/10.1177/0049124113500475 Cornish, T. (2016). Unconscious bias and higher education. Equality Challenge Unit. Available at: https://www.sdu.dk/-/media/files/om_sdu/fakulteterne/sundhedsvidenskab/ligestilling/unconscious-bias-and-higher-education1.pdf?la=da&hash=D7A481BBC4152581554EEBB6E8169ABBBA18F3AF Devine, P. G. (1989). Stereotypes and prejudice: Their automatic and controlled components. Journal of Personality and Social Psychology, 56(1), 5–18. https://doi.org/10.1037/0022-3514.56.1.5 Devine, P. G., Forscher, P. S., Austin, A. J., & Cox, W. T. L. (2012). Long-term reduction in implicit race bias: A prejudice habit-breaking intervention. Journal of Experimental Social Psychology, 48(6), 1267-1278. Fawcett Society (2017). Sounds Familiar? The Fawcett Society. Available at: https://www.fawcettsociety.org.uk/policy-research/soundsfamiliar/ Kahveci, A. (2010). Quantitative analysis of science and Chemistry Textbooks for Indicators of Reform: A complementary perspective, International Journal of Science Education, 32(11): 1495-1519. Levine, A., Park, J., & Kuo, H. J. (2020). Understanding Disability Biases in Undergraduate Rehabilitation Students: An Exploratory Study. Rehabilitation Counseling Bulletin. https://doi.org/10.1177/0034355220910238 Macdonald, A. (2014). It's not for people like me. London: Women in Science and Engineering. Martins, A. & Garcia, M. (2016). Between culture and the market: what do physics teachers take into account when choosing textbooks, IARTEM Brazil e-Journal, 7(1). Available at: http://biriwa.com/iartem/ejournal/volume7.1/papers/Paper2Martins_Garcia_Between_culture_and_the_market_IART EM_eJournal_7.1.pdf O’Connor, C. & Joffe, H. (2020). Intercoder Reliability in Qualitative Research: Debates and Practical Guidelines. International Journal of Qualitative Methods, 19: 1–13 Parkin, C. & Mackenzie, S. (2017). Is there Gender Bias in Key Stage 3 Textbooks? Content Analysis using the Gender Bias 14 (GB14) Measurement Tool. Advanced Journal of Professional Practice, 1(1): 23-40. Rose, S., Spinks, N. & Canhoto, A. (2015). Management Research: Applying the Principles. Abingdon: Routledge. Schwarz, N. (2000). Emotion, cognition, and decision making, Cognition and Emotion 14(4): 433–440. Sunar, S. (2011). Analysis of Science Textbooks for A Levels in the UK (1st Edition). Turkey: Middle East Technical University. Available at: http://www.esera.org/media/ebook/strand11/ebookesera2011_SUNAR-11.pdf The Institute of Physics (2014). Improving Gender Balance. London: Institute of Physics report. Accessed from: https://epws.org/improving-gender-balance-institute-physics-report/ The STEM Education Review Group (2016). STEM education in the Irish school system: A report on Science, Technology, Engineering and Mathematics Education. Dublin: Department of Education and Skills.