How can competencies in higher education be modeled and measured? – First milestones achieved in the new German research program KoKoHs

Author(s):

Susanne Schmidt(presenting / submitting)Christiane Kuhn(presenting)Olga Zlatkin-Troitschanskaia Sebastian Brückner

Conference:

ECER 2013

Network:

22. Research in Higher Education

Format:

Poster

Session Information

22 SES 05.5 PS, General Poster Session

General Poster Session

Time:

2013-09-11

12:30-14:00

Room:

FUAYE

Chair:

Contribution

Skills and competencies have become the backbone of economic prosperity in the knowledge society of the 21^st century. Even though international empirical research has recognized this fact, it has yet neglected modeling and measuring academic competencies in higher education (cf. Tremblay, 2012; Blömeke, Zlatkin-Troitschanskaia, Kuhn & Fege, 2013). The existing research deficit is partly due to the high complexity of defining academic competencies of graduate and PhD students in the face of the international and intranational diversity of study models, education structures, and teaching performances in the tertiary education sector. There do exist approaches to a structural stabilization of empirical higher education research (cf. Tremblay, 2013, on OECD’s feasibility study “Assessment of Higher Education Learning Outcomes” (AHELO)), as an analysis of the international state of research shows (cf. Kuhn & Zlatkin-Troitschanskaia, 2011). At the same time, these studies reveal the challenges for higher education research in measuring competencies in a valid and reliable way. Thus, there is an urgent need for appropriate instruments for modeling and measuring competencies in most university subjects.

This is why the German Federal Ministry of Education and Research (BMBF) initiated the research program “Modeling and measuring competencies in higher education (KoKoHs)” to create a systematic framework for about 70 individual projects in several domains of higher education http://www.kompetenzen-im-hochschulsektor.de/index_ENG.php). In this joint multi- and interdisciplinary research program experts from various disciplines are working together and networking nationally as well as internationally. The three main goals of the funding initiative are

to maintain the performance of the German tertiary education system in the face of growing international competition,
to contribute to international competence research in higher education, and
to create a framework for the evaluation of higher education performance so that policy decisions and institutional measures can be based on empirical evidence.

A coordination office has been installed in order to connect the funded projects to international research activities and in order to use synergies purposefully. The projects funded by KoKoHs cover important university subjects and domains, and have also been selected according to the quantitative criterion of the total number of students enrolled. Another conceptual trademark of KoKoHs is its focus on domain-specific and generic competencies. Following Weinert’s definition of competencies (2001), the competencies in KoKoHs include cognitive disposition, that is academically gained knowledge, as well as the motivational, volitional, and social disposition to apply the gained knowledge flexibly in different situations. With this in mind, the program is funding projects regarding the following subjects and generic areas: economic, social, educational, and engineering sciences, teacher education in STEM subjects, as well as general research competencies and self-regulation (being crucial academically taught generic areas).

Method

Instrument development and competence measurement require sustainable models based on theories that include the dimensionality and scaling of competencies (cf. Hartig, Klieme, & Leutner, 2008). The modeling needs to take into account superordinate institutional requirements and, in particular, the specific characteristics of a subject or domain (i.e. curricular aspects, content areas, situational embedding), but also the related cognitive requirements. The development of appropriate competence models for higher education relies on interdisciplinary cooperation as the only possibility to guarantee that the individual research will also link to national and international empirical education research. Apart from the development of competence models, KoKoHs also focuses on the methodology of competence assessment. This complex and multi-dimensional research area calls for equally com-plex and multi-dimensional research methods. First, statements need to be made on the dimen-sion, scaling, and development of generic and domain-specific competencies in order to create suitable measurement instruments accordingly. Then, the modeled competence constructs need to be adequately implemented as concrete measurement methods. This can be ensured by applying adequate psychometric measurement models, which should be able to assess not only personal (latent) features, but also situative requirements. Psychometric models that meet these re-quirements are based mainly on item-response theory (IRT).

Expected Outcomes

Given the deficits in higher education research in Germany, the KoKoHs research program contrib-utes to national systematization and stabilization. Its message is that theories, methods, and re-sults can dovetail with innovative national and, above all, international research activities. At the ECER conference, we would like to give an overview of the milestones achieved so far in the Ko-KoHs projects. In this regard, we will present examples of approaches to competence modeling in selected domains. The KoKoHs program is very interested in establishing and maintaining contacts with excellent in-ternational experts from different areas (primarily research methodology, teaching methodology, and psychology), particularly to invite them to participate in further scheduled KoKoHs events (e. g. as critical friends at the International PhD Workshop in November 2013, or as contributors at the Junior Faculty Research Conference in summer 2014 or the Final Meeting in fall 2014). The ECER conference offers an ideal platform on which to exchange research experiences and perspectives to foster stronger awareness of sustainable assessment processes in higher education.

References

Blömeke, S., Zlatkin-Troitschanskaia, O., Kuhn, C., & Fege, J. (Eds.) (2013). Modeling and meas-uring competencies in higher education. Rotterdam: Sense Publishers. Hartig, J., Klieme, E., & Leutner, D. (Eds.) (2008). Assessment of competencies in educational con-texts. State of Washington, Göttingen: Hogrefe & Huber Publishers. Kuhn, C., & Zlatkin-Troitschanskaia, O. (2011). Assessment of competencies among university stu-dents and graduates – Analyzing the state of research and perspectives. Mainz: Johannes Guten-berg University Mainz. Arbeitspapiere Wirtschaftspädagogik [working paper: business education], 59. Tremblay, K. (2013). OECD Assessment of Higher Education Learning Outcomes (AHELO): Ra-tionale, challenges and initial insights from the feasibility study. In S. Blömeke, O. Zlatkin-Troitschanskaia, C. Kuhn, & J. Fege (Eds.), Modeling and measuring competencies in higher educa-tion. Rotterdam: Sense Publishers. Trembley, K., Lalancette, D. & Roseveare, D. (2012). Assessment of Higher Education Learning Outcomes. Feasibility Study Report. Volume 1 – Design and Implementation. OECD. Weinert, F. E. (2001). Concept of competence: A conceptual clarification. In D. S. Rychen & L. H. Salganik (Eds.), Defining and selecting key competencies (pp. 45–65). Seattle, WA: Hogrefe & Hu-ber.

Author Information

Susanne Schmidt (presenting / submitting)

Johannes Gutenberg-University Mainz, Chair of Business Education

Christiane Kuhn (presenting)

Johannes Gutenberg University Mainz

Chair of Business Education

Mainz

Olga Zlatkin-Troitschanskaia

Johannes Gutenberg-University Mainz, Chair of Business Education

Sebastian Brückner

Johannes Gutenberg-University Mainz, Chair of Business Education

Search the ECER Programme

Search for keywords and phrases in "Text Search"
Restrict in which part of the abstracts to search in "Where to search"
Search for authors and in the respective field.
For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.

Session Information

Contribution

Method

Expected Outcomes

References

Author Information

Search the ECER Programme

Navigation

Info for