The Strengths and Difficulties Questionnaire (SDQ) is a brief and comprehensive instrument used to evaluate the mental health of children and adolescents. It includes 25 items, with 15 reflecting negative attributes and 10 highlighting positive ones. These items are divided into five specific dimensions, each consisting of five items: emotional problems (EP), conduct problems (CP), hyperactivity-inattention (HA), peer problems (PP), and prosocial behavior (PB). Each subscale is rated on a 3-point Likert scale ("Not True," "Somewhat True," or "Certainly True"). It can be administered quickly, applied across children and adolescents, and incorporates information from multiple informants. Freely available in over 90 languages, the SDQ offers a cost- and time-efficient method for assessing child and adolescent mental health across diverse populations (SDQinfo.com). A review by Kankaanpaa and colleagues (2023) revealed that the SDQ has particular problems with reliability and factor structure. It was found that some items (“obedient”, “agreement with adults”, “persistent”) weakened construct validity, while reverse-coded items could lead to measurement error. Besides these quantitative studies,there are limited studies examining how the SDQ is interpreted by this age group (11-17) and whether the problems with these items stem from comprehensibility. 

The conceptual framework of the study is based on the Cognitive Interview framework (Willis, 2005), which uses Tourangeau’s (1984) cognitive response model. According to this model, responding to a survey item is a complex four-stage process: 1-Comprehension, 2-Recall, 3- Judgment, and 4- Response Selection. This practical framework has been used to identify specific disconnects, particularly in the “comprehension” and “judgment” stages, such as whether students understand the terms or interpret the statements correctly. 

The research question of this study is “How adolescents aged 11-17 cognitively interpret the SDQ items and to what extent these subjective understandings correspond with the intended theoretical meaning of the scale?” Rather than assuming that the scale is universally understood correctly, the study focuses on whether there is an error in the literature. In this context, as part of a project aimed at revising the SDQ, this study also aimed to identify how these items are understood by students, possible word errors, ambiguities, and cultural incompatibilities using cognitive interview methods and to present these findings. 

Finally, considering that the SDQ is a widely used screening tool globally, addressing the psychometric inconsistencies reported in the literature is of great importance. It has been suggested that limitations such as low factor loadings or reliability issues may not only stem from statistical or translation errors but may also arise from a fundamental lack of comprehensibility and developmental appropriateness for adolescents. Such psychometric inconsistencies are thought to stem from participants’ inability to interpret the items as intended. Consequently, this study aims to demonstrate that identifying and addressing these ambiguities at the meaning level is a necessary prerequisite for enabling more reliable measurements and more accurate cross-cultural comparisons in the future.

This study employed a qualitative research design based on Willis’ (2005) Cognitive Interviewing model to deeply examine the “response process validity” of the SDQ. This approach focuses on the cognitive process involved in participants’ response formation. Thus, it allows for the identification of error sources independent of the structure being measured directly at their source, rather than inferring them from statistical outputs. Participants consist of adolescents aged 11-17, corresponding to the specific developmental stage for which the SDQ self-report form was designed. The pre-sample consists of 5 participants (3 boys and 2 girls) with an average age of 13.8. Participants were selected using a purposive sampling strategy to ensure diversity in terms of age and gender. In accordance with methodological standards for cognitive interviews, the final sample size is determined by the principle of ‘data saturation’ meaning that the data collection process will continue until no new types of errors or patterns of interpretation emerge. A review of the literature suggests that the final sample size for this type of study is expected to reach approximately 10 to 12 participants. The primary data collection tool used was the self-report form of the SDQ. Interviews were conducted online (via Zoom) using a semi-structured protocol. The ‘Immediate Verbal Probing’ technique was used in the study. In this procedure, the adolescent was asked to read and respond to a specific item; immediately after the response, the researcher posed probes (e.g. ‘Why did you choose that option?’ or ‘What does this word mean to you?’) before moving on to the next question. This sequential approach, which included probing immediately after response was chosen to minimize ‘recall bias’. Data analysis followed a deductive qualitative approach; notes were taken during all interviews and analyzed using a schema appropriate for the target age group, based on Willis’(2005) Cognitive Interview Framework. The responses obtained were classified into four diagnostic categories according to this schema: ‘Structural Validity Inconsistencies’, where the student’s subjective interpretation deviates from the targeted theoretical structure; ‘Structural Errors’ that express the cognitive conflict created by double-sided items; ‘Expression and Comprehension’ problems caused by words that are not appropriate for the developmental level; and ‘Social Desirability Bias’ that causes self-censorship due to fear of judgment.

According to data obtained from cognitive interviews, issues related to scale items were grouped under four headings: Construct Validity Mismatches, Double-Barreled Items, Wording & Comprehension, and Social Desirability Bias. Initial results showed that two items (10 & 3) were perceived differently by students from the meaning the scale intended to measure. At the same time, it was observed that, especially in items consisting of two sentences (12,16 & 25), the two statements did not measure the same situation and were evaluated as having different meanings (Double-barrelled). From a language and expression perspective, it was seen that the words in some items (11,19&23) were not age appropriate and created confusion for students. Finally, two items (18 & 22) were found to be disturbing by students, and it was stated that they could not be answered honestly. Cognitive interviewing has been emphasized as an important methodological step in the adaptation and evaluation of the SDQ, particularly for identifying item-level problems that may not be detectable through quantitative analyses alone ( Goodman et al., 2010). In line with previous SDQ studies, the present findings indicate that certain items were interpreted in ways that diverged from their intended constructs, and that double-barreled wording, linguistic complexity, and socially sensitive content posed challenges for respondents. Similar concerns have been reported in SDQ validation studies across different cultural contexts, where item ambiguity, wording effects, and social desirability have been shown to affect item functioning and factor structure (e.g., Van Roy et al., 2008; Ortuño-Sierra et al., 2015; Karlsson et al., 2022). Together, these results reinforce the view that item-level validity issues in the SDQ may contribute to the inconsistent factor structures reported in the literature and underscore the necessity of incorporating cognitive interviews as a preliminary step before drawing substantive conclusions from SDQ scores.

Goodman, A., Lamping, D. L., & Ploubidis, G. B. (2010). When to use broader internalising and externalising subscales instead of the hypothesised five subscales on the Strengths and Difficulties Questionnaire (SDQ): Data from British parents, teachers and children. Journal of Abnormal Child Psychology, 38(8), 1179–1191 Kankaanpää, R., Töttö, P., Punamäki-Gitai, R.-L., & Peltonen, K. (2023). Is it time to revise the SDQ? The psychometric evaluation of the Strengths and Difficulties Questionnaire. Psychological Assessment, 35(12), 1069–1084. Karlsson, P., Larm, P., Svensson, J., & Raninen, J. (2022). The factor structure of the Strengths and Difficulties Questionnaire in a national sample of Swedish adolescents: Comparing 3- and 5-factor models. PLoS ONE, 17, e0265481. Ortuño-Sierra, J., Chocarro, E., Fonseca-Pedrero, E., Rivas, S., & Muñiz, J. (2015). Screening mental health problems during adolescence: Psychometric properties of the Spanish version of the Strengths and Difficulties Questionnaire. Journal of Adolescence, 39, 49–57. Tourangeau, R. (1984). Cognitive science and survey methods. In T. B. Jabine, M. L. Straf, J. M. Tanur, & R. Tourangeau (Eds.), Cognitive aspects of survey methodology: Building a bridge between disciplines (pp. 73–100). National Academy Press. Van Roy, B., Veenstra, M., & Clench-Aas, J. (2008). Construct validity of the five-factor Strengths and Difficulties Questionnaire (SDQ) in pre-, early, and late adolescence. Journal of Child Psychology and Psychiatry, 49(12), 1304–1312. Willis, G. B. (2005). Cognitive interviewing: A tool for improving questionnaire design. Sage Publications