COMPES: Development of an Instrument for Basic-Competencies Assessment of Students in Higher Education

Session Information

22 SES 02 B, Teaching, Learning and Assesment in Higher Education

Paper Session

Time:
2010-08-25
11:15-12:45
Room:
M.B. SALI 15, Päärakennus / Main Building
Chair:
Elinor Edvardsson Stiwne

Contribution

 

The constant need of adaptation to the fast-moving social and economical changes produces an uncertainty that has become a characteristic of the current professional world. European universities, through the European Higher Education Area want to promote the development of transferable and adaptable competencies for life-long learning (Whitston, 1998; Fallows y Steven, 2000) such as creative problem-solving, search of relevant information, development of critical thinking or autonomous learning (Birenbaum, 1996; Segers y Dochy, 2001).

 

From an assessment viewpoint, practices should be adapted to the objectives to achieve. Therefore, from some time ago it is being developed the learning-oriented assessment. It has the aim to use assessment as a way to develop competencies through: authentic tasks, students involvement in assessment and providing feedforward (Carless, 2003;  Boud y Falchikov, 2006; Keppell et al., 2006; Bloxham y Boyd, 2007; Rodríguez, Ibarra y Gómez, in press).

 

On the other hand, the digital era has expanded to all the professional fields. Teachers should be capable to give learning opportunities through the use of ICT for university students’ assessment. This situation has resulted in the e-Assessment, understood as “The end-to-end electronic assessment processes where ICT is used for the presentation of assessment activities and the recording of responses. This includes the end-to-end assessment process from the perspective of learners, tutors, learning establishments, awarding bodies and regulators, and the general public” (JISC, 2007:6).

 

When we focus on learning oriented-assessment in virtual contexts we can talk about e-Learning oriented e-Assessment (Rodríguez et al., 2009). To study this concept further, EvalFOR group, from the university of Cádiz, is working in a research project, named Re-Evalúa “Re-engineering of e-assessment, technologies and development of teachers and students’ competencies” financed by the Junta de Andalucía (reference P08-SEJ-03502).

 

The main objective of Re-Evalúa is to test empirically the results obtained with the incorporation of the “e-Learning oriented e-Assessment” to the teaching and learning process in:

a)      The development of university teachers’ competencies.

b)      The development of basic competencies regarding assessment in university students.

 

To achieve these objectives, a “quasi-experimental” methodology has been designed, with a pre-test post-test design with quasi-control groups. The independent variable is the introduction of e-learning oriented e-assessment and as dependent variables the development of teaching competencies and basic competencies in students assessment.

 

Concerning the development of students’ competencies, an instrument for the evaluation of students’ perceptions about their own competencies before and after participating in an e-Learning oriented e-Assessment course has been designed. This instrument is called COMPES (“Self-report about the Development of Basic Competencies in University Students”). The aim of the instrument is to obtain valid and reliable information about whether the e-Learning oriented e-Assessment is a strategy that develops university students competencies.

 

In this paper COMPES design and validation process is described as a tool for students’ reflection and self-assessment, as instrument of assessing teacher  and also as an instrument for pre-test, post-test data gathering.

Method

A. PANEL OF JUDGES VALIDATION. A1. Identification of the basic competencies in assessment. The initial proposal is based in the Dublin descriptors. A2. Definition of basic competencies. With an on-line questionnaire the clarity, validity, relevance, usability and connection with assessment are evaluated. 36 researchers participated. A3. Description of performances related to the competencies. Each competence is divided in performances. Through an on-line questionnaire the coherence with the definition is evaluated. 50 researchers participated. A4. Creation of COMPES Matrix. Competencies, definitions and performances are described. A frequency scale with four levels is attached to each performance. A5. Creation of COMPES Self-report. The performances are written as affirmative sentences randomly organized. Each affirmation has a 1 to 6 rating scale. B. EMPIRICAL VALIDATION B1. Pilot study of COMPES Self-report. 55 students participated. B2. Revision and conclusions of the pilot. In a meeting, the results obtained are analyzed and COMPES is modified.

Expected Outcomes

After the validation process, 10 basic competencies are identified, 7 of them with 4 performances and 3 competencies with 3 performances. The level of each competence is defined as the frequency in which students use these performances. The 10 basic competencies are: 1. Knowledge application. 2. Argumentation 3. Problem solving 4. Information analysis 5. Communication 6. Autonomous learning 7. Ethic sense 8. Creativity 9. Work in groups 10. Assessment/Evaluation As an example we show a performance of competence 1. Knowledge application: “Implement the knowledge correctly in specific tasks” In COMPES self-report performances are written as affirmative sentences. At the moment, a test with 250 students is being made at the University of Cádiz to guarantee its fiability before being used for thousands of students in the nine Spanish universities participating in Re-Evalúa project. COMPES theoretical construction and empiric validation has been a collaborative process between experts leading from EvalFOR research group. We want to highlight that in the last meeting it was decided that COMPES Matrix would be adapted to the students and that complementarily a comparison would be made to evaluate the applicability that it has as an alternative to the Self-report format.

References

- BIRENBAUM, M. (1996). Assessment 2000: towards a pluralistic approach to assessment. En M. BIRENBAUM Y F.J.R.C. DOCHY (Eds.). Alternatives in assessment of achievements, learning processes and prior knowledge (3-30). Boston, MA: Kluwer Academic Publisher. - BLOXHAM, S., Y BOYD, P. (2007). Developing Effective Assessment in Higher Education. A Practical Guide. New York: Open University Press - MCGraw Hill Education - BOUD, D. Y FALCHIKOV, N. (2006). Aligning assessment with long-term learning. Assessment & Evaluation in Higher Education, 31(4), 399-413. - CARLESS, D. (2003). Learning-oriented assessment. Paper presented at the Evaluation and Assessment Conference, University of South Australia, Adelaide, 25 de noviembre de 2003 - FALLOWS, S.J. Y STEVEN, C. (2000). Building employability skills into the Higher Education curriculum: a university-wide initiative. Education and Training, 42, 75-82. - JISC (Joint Information Systems Committee) (2007). Effective Practice with e-Assessment. An overview of technologies, policies and practice in further and higher education. Available at http://www.jisc.ac.uk/media/documents/themes/elearning/effpraceassess.pdf. - KEPPELL, M., AU, E., MA, A. Y CHAN, C. (2006). Peer learning and learning-oriented assessment in technology-enhanced enviroments. Assessment & Evaluation in Higher Education, 31(4), 453-464. - RODRÍGUEZ GÓMEZ, G., IBARRA SÁIZ, M.S., DODERO BEARDO, J.M., GÓMEZ RUIZ, M.A., GALLEGO NOCHE, B., CABEZA SÁNCHEZ, D., QUESADA SERRA, V. Y MARTÍNEZ DEL VAL, A. (2009). Developing the e-Learning-oriented e-Assessment. V Internacional Conference on Multimedia and Information and Communication Technologies in Education. Lisboa: Formatex, 515-519. - RODRÍGUEZ, G., IBARRA, M.S. Y GÓMEZ, M.A. (in press). e-Autoevaluación en la universidad. Un reto para profesores y estudiantes. Revista de Educación. - SEGERS, M. Y DOCHY, F. (2001). New assessment forms in problem-based learning: the value-added of the students’ perspective. Studies in Higher Education, 26(3), 327-343. - WHITSTON, K. (1998). Key skills and curriculum reform. Studies in Higher Education, 23, 307-331.

Author Information

Universidad de Cádiz
Departamento de Didáctica
Puerto Real (Cádiz)
University of Cadiz
EVALfor Research Group
PUERTO REAL
Universidad de Cádiz, Spain
Universidad de Cádiz, Spain

Update Modus of this Database

The current conference programme can be browsed in the conference management system (conftool) and, closer to the conference, in the conference app.
This database will be updated with the conference data after ECER. 

Search the ECER Programme

  • Search for keywords and phrases in "Text Search"
  • Restrict in which part of the abstracts to search in "Where to search"
  • Search for authors and in the respective field.
  • For planning your conference attendance, please use the conference app, which will be issued some weeks before the conference and the conference agenda provided in conftool.
  • If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.