Research Question In “PISA 2003 international”, problem solving as one of the “cross-curricular-competencies” is divided into analytic and dynamic problem solving. For assessing analytic problem solving “paper and pencil tests” are used, whereas computer based tools are necessary to assess dynamic problem solving. The project is targeted at the development and the empirical evaluation of a competence model for dynamic problem solving, and at the ascertainment of the curriculum part of this cross-curricular competence. The research project’s model is restricted to cognitive aspects and focussed on a domain specific topic of the chemistry curriculum: ester and the chemical equilibrium. Theoretical framework The last large scale assessments like the TIMS-Study or the “Program for International Student Assessment” (PISA) show very clearly that most of the OECD countries have problems regarding the efficiency of their science education in schools. Consequently, many of these countries like e.g. the USA, Great Britain or Germany have developed national education standards (e.g. Carter, 2005). And although a lot of reforms for implementing these standards take place in education policy, only little research is carried out about the efficiency of these reforms (e.g. Lemke, 2001). In this research, where problem solving abilities have to be evaluated especially in chemistry, a competence model is necessary to design appropriate tasks. This study will contribute fundamentally to overcoming theoretical defeciencies. Competence Following the most influential definition of competence by Weinert (1999), competencies are defined as available or learnable cognitive abilities or skills as well as the willingness of a person to solve certain problems and to apply the solution successfully and in a responsible way (s.a. Franke, 2005. Hence, competencies can be described as context-sensitive, cognitive dispositions of accomplishments connected with specific demands and situations. The correspondant scopes as well as professional and educational aspects have to be considered while developing appropriate competency models. However, competency descriptions often are obtained post hoc, and theoretical frameworks are available only for some topics of the school curriculum. Problem solving has been defined in a variety of ways, and a vast number of studies during the past 20 years has documented the research interest into this topic (Tiemann, 1999, 2001). Problem solving In general a problem exists when a student perceives a gap between an actual situation and a situation he or she wants to have but doesn’t know how to bridge the gap (Dörner, 1987; Hayes, 1981). So whether a “gap” really is a problem or only a task strongly depends on the student’s prior knowledge and skills. Therefore the competence to solve a problem is defined as “an individual’s capacity to use cognitive processes to confront and resolve real, cross-disciplinary situations where the solution path is not immediately obvious and where the literacy domains or curricular areas that might be applicable are not within a single domain of mathematics, science or reading” (OECD 2003, p. 156). Within PISA, problem solving is described by the following steps (OECD 2004, p. 27): (1) understanding the problem (PU), (2) characterizing the problem (PC), (3) representing the problem (PR), (4) solving the problem (PS), (5) reflecting on the solution (SR) and (6) communicating the solution (SC). These steps have to be considered, if tasks for analyzing problem-solving competence are designed. In science education only some of these steps have been described and sufficiently studied (e.g. Bunce, Gabel & Samuel, 1991; Gabel & Bunce, 1994; Fraser, 2003; Chinn & Brewer, 2003).

This study combines various instruments and methods (table 1): Evaluated questionnaires of other science education research projects are used to control motivation and interests, also for the assessment of general background data, computer familiarity or fundamental cognitive abilities. The domain specific prior knowledge is controlled by an instrument developed in this study. In order to test the dynamic problem solving competence a computer based assessment tool is designed and evaluated. This tool gives information about the content extent and the structural organization of the competence dimensions. Tab. 1.: Overwiew about the instruments and methods _____________________ 1st day Constructs: prior knowledge, cognitive basic skills, interest and motivation, general background data, familiarity with computers Instrument/method: questionnaires (multiple choice, short tasks) 2nd day Construct: dynamic problem solving Instrument/method; computer based assessment tool _____________________ Sample/ Design At first, the cells of the matrix, built up by the steps of problem solving and the three levels of competencies, are described in detail. These general descriptions, without any chemical contents, are assigned again to the cells by 20 students with little background knowledge in science. In the next step, the descriptions of the requested cell features are based on chemical contents. Using a four-stage Likert scale chemistry teachers (n=20) should decide, if a described aspect should be related to the stated step of the problem-solving process or not. Also 28 students becoming chemistry teachers are asked to assign the descriptions? to the cells of the matrix. Based on the results of these pre-studies, a computer based assessment tool is designed. It is a “virtual laboratory”, where the students have to solve problems concerning the different steps of the problem solving process. These problems can be solved on different levels and are part of a main problem, the optimized syntheses of an ester. This virtual laboratory, now clearly identifying the different steps and levels, is tested for usability in classroom in a further pre-study with 128 10th grade students at grammar schools in Berlin (Germany). The main study is carried out with 370 students, again 10th grade at grammar school in Berlin. To expect a middle effect size for interaction effects, this sample is suitable on the alpha=.01 level.

The study began in April 2006 and will last until February 2009. Now a short overview will be given about the results of the pre-studies. The study investigating the seperability of each problem-solving step (questionnaire A) showed that especially the steps “understanding the problem”, “characterizing the problem” and “solving the problem” could not be seperated clearly. For these three steps, the agreement with the model is relatively small (understanding the problem 55.9 %, characterizing the problem 58.5 % and solving the problem 58.3 %). The other three steps show a good agreement with the model (“representing the problem”: 70%, “reflecting on the solution”: 77.5% and “communicating the solution”: 74.4%). A remarkable finding is that many test persons related statements actually belonging to the step “understanding the problem” to the step “characterizing the problem” and vice versa. By combining these two steps an agreement of 79.3 % is achieved. In the case of “solving the problem” no similar effect could be noticed, hence a combination with another step is not applicable. The steps “reflecting on the solution” and “communicating the solution” were also combined. step reliability number of items PU 0,91 PC 0,86 10 PR 0,82 5 PS 0,88 6 SR 0,89 SC 0,88 10 Table 2: Results of the chemistry-specific description of the problem-solving steps For characterizing the content of the single cells of the model, the other two questionnaires (B, C) were used. In order to detect which activities are important for each step when solving a problem in chemistry (questionnaire B), the reliability of the analyzed problem-solving steps were determined. Table 2 shows a survey of the results. By means of questionnaire C, using a multi-stage procedure, all descriptions of those levels which could not be clearly associated to one specific level of competency were acquired. The assessment tool for dynamic problem solving in chemistry works well. It could be expected that teaching and learning chemistry in schools foster special aspects in acquiring the competence of dynamic problem solving: The specific ways in chemistry of thinking, of arguing and of planning and carrying out experiments play a unique role for this competence. Further studies with this instrument will investigate the influence of different curricular topics and the development of problem solving competency.