A Programme of Educational Leave - Evaluation Problems and Strategies

Author(s):

Lorenz Lassnigg(presenting / submitting)

Conference:

ECER 2012

Network:

02. Vocational Education and Training (VETNET)

Format:

Paper

Session Information

02 SES 11 C, Educational Leave, Intergenerational Learning and Social Capital

Parallel Paper Session

Time:

2012-09-20

17:15-18:45

Room:

FCT - Aula 23

Chair:

Martin Mulder

Contribution

The paper reflects on a project that was conducted in Austria for the evaluation of an educational leave programme called ‘Bildungskarenz’. The main purpose is to put the measure and the results into an international perspective, and to discuss some methodological issues that have come up during the project. First the measure, the evaluation methodology, and the results will be presented; second, the project will be put into a comparative perspective, by asking which similar measures and evaluations can be found in the research literature; third, some major problems will be discussed that arise with the intention of a ‘rigorous’ evaluation of this kind of measure.

The ‘Bildungskarenz’ is a measure provided in Austria for more than 10 years; it aims at giving adult employed persons the opportunity to attend extended programmes of full-time learning and education, by giving them support for their living expenses for a longer period of time, i.e. up to one year. The parameters have changed in the course of provision, however, main points are:
- the attendants must have been employed for a longer period of time before starting the leave;
- the employer must agree, and should not dismiss someone on grounds of the leave;
- the measure is administrated by the public employment service, and the individual support is financed by the unemployment insurance fund
- the support has been low in the beginning and increased to the amount of the entitled unemployment compensation of the person
- the participants have to prove some hours of attention of education or training programmes.

A very broad approach of evaluation has been commissioned, however, with a given cap on the financing of the project (see next section for the methodology).

The evaluation was financed by the Ministry of Labour, and the questions to be answered were strongly focused on a ‘rigorous’ evaluation of the labour market outcomes of the leave for the participants. Besides that several other questions, as the positioning of the programme in the overall adult education system of Austria (e.g., share of participation), the structure of the participants and their participation (e.g., employment and personal characteristics, time periods, ET programmes taken; success; outcomes after participation), and the subjective assessment of their participation by the individuals have been analysed. Main interests have been first the economic benefits of the programme, and second how far the measure was able to support less educated and less qualified employees in order to overcome the well known ‘Matthew Effect’.

Preliminary results at the international level show that programmes of this kind have been conducted in other countries also; however, such a broad evaluation has not been conducted elsewhere. This had two pitfalls for the project: first, we could not rely on previous experience of other projects; second, the programme is not really compatible with the very well established methodology of evaluation of labour market policy programmes; the latter has nevertheless very much guided the commissioning and the interest of the Ministry as the commissioning body.

Method

The methodology of evaluation was based on two cornerstones, a survey among participants from different years, and an analysis of employment records of participants before and after the measure and a matched comparison group from the available statistical data bases; the surveyed participants were identified in the statistical data basis to obtain their ‘objective’ employment record. Because of the financing cap, the enterprises could not be surveyed, so information about applicants, who wanted but did not participate, was not available. Three big questions were addressed: - First, how does the measure fit into the landscape of adult education in Austria, and how are the participants and their ET programmes positioned in relation to the overall structure of participation? Analysis was based on a comparison of the participants with population surveys (LFS and AES) - Second, how does the implementation of the ‘Bildungskarenz’ work, and how do participants evaluate the measure? Analysis was based on participants survey - Third, how are the economic benefits of the measure in terms of employment career and income? Analysis was based on statistical data about employment records that include detailed information about employment and unemployment spells through the life course, and related income.

Expected Outcomes

Main results are: - employers did not utilise the measure before the crisis of 2008/09, this seemed to change afterwards (similar experience from Netherlands); - higher propensity among participants to participate in adult education than among non-participants; very low educated poorly reached; - participants’ subjective evaluation is very favourable across the board; - the statistical evaluation of economic benefits obtained ceteris paribus not benefits but losses of participants vis a vis the comparison group for a period up to seven years. Other studies about similar measures in other countries obtain similar tendencies: very favourable subjective evaluations, and no or little ‘objective’ economic benefits. Main problems that have arisen from the evaluation, and to be addressed in the paper are the following: - the goals of the measure were wide and unclear, support to lifelong learning and flexibility, incentives for enterprises’ training interest, individual improvement; among these the focus on economic benefits seems misguided - established practices of labour market policy evaluation geared to unemployed are not feasible for the analysis of this measure (flawed matching procedure); - a very much smaller qualitative evaluation of the same measure in Austria has obtained very similar results.

References

Selected Rferences: Andersen, D.; A. Appeldorn and H. Weise (1996) Evaluation of the Danish Leave Schemes. Summary of a Report [microform] / http://www.eric.ed.gov/PDFS/ED410382.pdf CEDEFOP (2011) Training leave and payback clauses: policies and practices in Europe http://www.cedefop.europa.eu/EN/events/18265.aspx CEDEFOP (forthcoming) Training leave to promote VET in Europe. Report of a comparative European project (unpublished draft April 2011). Card, D.; J. Kluve and A. Weber (2009) Active Labor Market Policy Evaluations: A Meta-Analysis. IZA DP No. 4002 DISCUSSION PAPER SERIES, Bonn: Institute for the Study of Labor Diller, F.; Festner, D.; Freiling, Th. et al. (2011): Qualifikationsreserven durch Quereinstieg nutzen. Studium ohne Abitur, Berufsabschluss ohne Ausbildung Bielefeld: W. Bertelsmann Verlag IZA-Bonn (2012) Evaluation of Labor Market Programs, webpage http://www.iza.org/en/webcontent/research/ra1/viewRA?lang=en Kristensen, Nicolai and Lars Skipper (2009) Impact analyses of adult education, re-education and further education English Summary Analysis of the impact on individuals and a cost-benefit analysis København: AKF Anvendt KommunalForskning http://www.akf.dk/udgivelser/2009/pdf/effektanalyser_af_voksenefteruddannelse.pdf Lassnigg, L.; Gottwald, R.; Hofer, H.; Kuschej, H.; Zaussinger, S. et al. (2011): Evaluierung der Bildungskarenz 2000-2009. IHS research report. Vienna: IHS. Wolf, A.; Jenkins, A. and Vignoles, A. (2006): Certifying the workforce: economic imperative or failed social policy? Journal of Education Policy, Vol. 21, No. 5, September 2006, pp. 535–565 DOI: 10.1080/02680930600866124

Author Information

Lorenz Lassnigg (presenting / submitting)

Institute for Advanced Studies (IHS)

Sociology

Vienna

Search the ECER Programme

Search for keywords and phrases in "Text Search"
Restrict in which part of the abstracts to search in "Where to search"
Search for authors and in the respective field.
For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.