Similarities And Differences Between Two Types Of External Evaluation For Higher Vocational Colleges And Resulting Long-term Benefits For The College

Author(s):

Lidija Grmek Zupanc(presenting / submitting)

Conference:

ECER 2016

Network:

11. Educational Improvement and Quality Assurance

Format:

Paper

Session Information

11 SES 09 B, Paper Session

Paper Session

Time:

2016-08-25

13:30-15:00

Room:

OB-E0.32

Chair:

Heidi Flavian

Contribution

The article discusses external evaluation at a Slovene higher vocational college – more precisely, two types of external evaluation. The first type is mandatory for higher vocational colleges and the second is voluntary. The main research question focuses on the similarities and differences between the two types of external evaluation. Considering our recent personal involvement as the head teacher of a higher vocational college in both types of external evaluation, the analysis of similarities and differences using documents and literature will cover the period from the very beginning, i.e. from application to final separate reports of each type of external evaluation.

The Slovenian Quality Assurance Agency for Higher Education (SQAA) is tasked with carrying out mandatory external evaluation of higher education institutions.

Article 2 of the Criteria for the External Evaluation of Higher Vocational Colleges (Merila za zunanjo evalvacijo višjih strokovnih šol) stipulates that in performing its tasks, SQAA also takes into account the Standards and Guidelines for Quality Assurance in the European Higher Education Area.

Article 6 of the Criteria for the External Evaluation of Higher Vocational Colleges provides that the Agency shall carry out external evaluation of the higher vocational college every five years (scheduled external evaluation) and the procedure ends with a decision on the achievement of the statutory standards defined by these criteria. However, external evaluation can also be extraordinary.

Article 7 thereof determines the purpose of external evaluation. It is used to establish the achievement of the statutory standards defined by these criteria, progress in the performance of the higher vocational college and improvement in the provision of study programmes in all areas defined in Article 8 thereof.

Article 8 provides the areas of assessment of the quality of the higher vocational college: 1. integration with the environment, 2. functioning of the higher vocational college, 3. human resources, 4. students, 5. material conditions (premises, equipment, library-information activity and funding), 6. quality assurance.

Because we had been preparing thoroughly for the scheduled external evaluation for quite a while, we also applied for voluntary external evaluation for business excellence before the scheduled evaluation. The scheduled assessment went well and no disparities were observed. We were commended for a number of aspects in the preliminary oral report. To this end, we were glad to have applied for the voluntary external evaluation and thus became one of the first colleges in Slovenia to take that step. We applied for the Slovenian Business Excellence Prize, which is awarded by the Ministry of Economic Development and Technology of the Republic of Slovenia. We were aware of the fact that it was a question of prestige and we were willing to undergo such an assessment. Prior to the assessment, we organised a seminar at our school for the leadership and the members of the quality commission; each participant was also given the book EFQM Excellence Model. On page 2 it states that the EFQM Foundation was established with a view to recognising and promoting sustainable excellence and providing guidelines to those wishing to attain it. The Foundation pursues this goal through three intertwined components: the fundamental principles of excellence, the EFQM Excellence Model and the RADAR Logic. The following 5 criteria are also important: 1. Leadership, 2. Strategy, 3. People, 4. Partnerships & Resources, 5. Processes, Products & Services. Each criterion is divided into five parts (a, b, c, d and e): creating added value for the customer, creating a sustainable future, developing organisational capabilities, nurturing creativity and innovation, leading with vision, inspiration & integrity, managing with agility, succeeding through the talent of people, and sustaining outstanding results.

Method

"Despite differences in the scope and range of activities evaluated, the implementation of external school evaluation across Europe is based on a highly homogenous structure which consists of three basic steps: 1. analysis, 2. visit, 3. reporting" (2015, page 8). Our goal is to include all three elements in the comparison. 1. Comparing documents that had been sent prior to the scheduled and the voluntary external evaluation We were required to submit a number of college documents before the scheduled evaluation. As for the voluntary external evaluation, we devised a separate Management Document in line with the guidelines. Some of the information was summarised from school documents, while other information was provided according to the requirements. In speaking of the EFQM Model, it immediately becomes clear that its terminology is intended for the economy, e.g. the customer. Such expressions were then "translated" to fit the field of education, meaning that customers in higher vocational colleges are students. 2. Comparing timetables for the scheduled and the voluntary external evaluation The school hosted the group of experts for two days. External evaluation only covered the higher vocational college and not the remaining units of the school centre. The group of experts consisted of the chairperson and two members. They first held an introductory meeting with the leadership and commission chairs and then visited the college, its library, and students in lecture halls. Moreover, they interviewed the quality commission, the employees of the student affairs office, the organisers of practical education, mentors appointed by the companies hosting our students on internship, employers, and students. On the second day, the group of experts visited some of the employers. As for the timetable for the business excellence assessment, it spanned three days because the entire school centre, i.e. the higher vocational college as well as three high schools, was subject to the assessment. A total of four external assessors were involved, namely its chair and three members. First, they held a presentation for the leadership, examined the organisation, and conducted interviews. 3. Comparing reports we received following the scheduled and the voluntary external evaluation The comparison between the SQAA report and the MIRS report (Metrology Institute of the Republic of Slovenia – the Slovenian Business Excellence Prize) suggests that the structure of both reports is similar in that they consider the criterion parts and first list the findings followed by the advantages and our opportunities.

Expected Outcomes

At this point, it is necessary to consider the benefits, i.e. the "added value" of double external evaluation within two months. If we focus on written reports alone, then both reports need to be analysed in detail. Of course, we may look for similarities and differences between the criteria of the scheduled external evaluation and the voluntary external evaluation (i.e. similar requirements). However, it is more important to analyse the findings and opportunities for improvement provided by both reports to achieve the expected results. It would also be sensible to divide the findings into three groups. The first group would encompass similar findings in terms of content. The second group would involve the findings that share some overlapping elements, while the third group would contain the findings that were only found in one of the reports. The same system could be used in terms of opportunities for improvement. The first group would list the opportunities for improvement with similar content. The second would summarise the findings with several overlapping characteristics, while the third group would contain the opportunities that do not overlap at all. The expected goal is thus to compile the first group of findings and the first group of opportunities for improvement. They would be extracted from the reports the assessors produced according to slightly different criteria. Another expected goal is to work on the first group of opportunities for improvement because they should actually become the major priorities of the school centre. Although the application for the business excellence assessment is about prestige and rising above average, each report and especially their overlapping features, findings and opportunities for improvement have a special meaning for the school centre.

References

Criteria for the External Evaluation of Higher Vocational Colleges. [http://test.nakvis.si/en-GB/Content/Details/253] available [Accessed: 5.1.2015] European Commission/EACEA/Eurydice. (2015). Assuring Quality in Education: Policies and Approaches to School Evaluation in Europe. Eurydice Report. Luxembourg: Publications Office of the European Union. EFQM Excellence Model, Excellent Organizations achieve and sustain outstanding levels of performance that meet or exceed the expectations of all their stakeholders. (2013). EFQM, Avenue des Olympiades 2, 1140 Brussels, Belgium. Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG). (2015).

Author Information

Lidija Grmek Zupanc (presenting / submitting)

SCHOOL CENTRE KRANJ

VOCATIONAL COLLEGE

KRANJ

Search the ECER Programme

Search for keywords and phrases in "Text Search"
Restrict in which part of the abstracts to search in "Where to search"
Search for authors and in the respective field.
For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.