Evaluating the net treatment effects of a continuing vocational training programme (evidence from Slovakia)

Author(s):

Miroslav Štefánik(presenting / submitting)

Conference:

ECER 2014

Network:

09. Assessment, Evaluation, Testing and Measurement

Format:

Paper

Session Information

09 SES 08 B, Assessing Expectations Towards and Achievements in Vocational Education and Training

Paper Session

Time:

2014-09-04

09:00-10:30

Room:

B012 Anfiteatro

Chair:

Antra Ozola

Contribution

Substantial attention of social researchers is oriented at evaluation of net treatment effects of training programmes. Thanks to this attention, evaluation methods have improved significantly since the Eighties. Most of the troubles in evaluating training programmes arise because of the problems related to data gathering using experiments in educational studies. Big effort in this field was done in microeconomics, where a plenty of non-experimental evaluation methods were developed. One of the options available here is an ex-post selection of the control group and a consequent comparison of outcomes of those who received the training or education programme with the members of the control group.

These methods became especially popular in quantitative policy evaluation in recent years. They are suited in situations when the goal of a measure is clearly defined and can be measured using a limited number of indicators.

In case of our paper we are going to evaluate a training programme which is a part of the set of active labour market measures in Slovakia. It is targeting registered unemployed from the beginning of their unemployment. It offers coverage of expenses related to all trainings improving individuals’ employability. The offer of training courses is strongly determined by the regional centre administrating the measure. We therefore perform the analysis with respect to region. Evaluated programme was recognised as continuing vocational training programme also by the European Commission (for example in: (European Commission, 2013)).

The target of the programme is defined clearly: to improve the employability of participants and thus improve their chances to find a job. Thanks to this, a reliable outcome indicator can be easily constructed. It is the proportion of those who left the unemployment registers due to placement into a job. This proportion can be compared between programme participants and the control group.

Studies evaluating training measures, using comparable methodology, report positive effects of training on future changes of getting a job (Blundell, et al., 2004) (Lechner, et al., 2007) (Bořík, et al., 2013). Especially in long-term there is strong evidence on vocational training programmes contributing to employability of individuals. This was only partially confirmed for Slovakia.

Method

Methods based on ex-post selection of the control group are referred to as to the contra - factual evaluation as the outcomes of (training/education) programme participants are compared to outcomes of a contra factual situation. Heckman shows that the problem of contra factual evaluation on non-experimental data is the problem of missing data, when the information on the outcome of participants, if they would not participate, is missing (Heckman, et al., 1997). This problem is solved by the ex-post control group approach using the propensity score matching method under two assumptions (of unconfoudedness and the assumption of overlap) (Caliendo, et al., 2005). The matching procedure was done by exact matching on region, education level, gender and age group of the participants. Within each of this subgroups matching was done using the nearest neighbour matching using the propensity score variable. To calculate the propensity score variable a logit function on the participation variable was applied, other characteristics such as field of education, number of children, declared skills or unemployment duration were included as explanatory variables.

Expected Outcomes

Expected positive effect of the programme was confirmed only partially on Slovak data. When looking on the country average effect of the programme we, surprisingly, get negative figures. Those who participated in the training were less successful in finding a job, than the members of the control group. After splitting the analysis based on regions we have found out that the effect of the measure differs regionally. As the offer of training courses is defined on the regional level this could be expected. The biggest difference is between evidence from Bratislava and the rest of the country. In Bratislava half of the total trainings are offered. Their effects are significantly negative, which is in contrast to positive effects measured in the dominant part of the rest of the country. There are no significant differences in (control group vs. treatment group) balance between the data from Bratislava and the rest of the country. Therefore the method seems to be reliable, differences in implementation and other socio-economic differences between regions play a role. Until September our current results from the propensity score matching approach will be compared to figures acquired by an instrumental variable approach, to acquire additional support for our findings. (See (Caliendo, et al., 2005) for the classification of the methods)

References

Blundell, Richard, Dearden, Lorraine and Sianesi, Barbara. 2004. Evaluating the Impact of Education on Earnings in the UK> Models, Methods and Results from the NCDS. Centre for Economics of Education. Working paper, 2004, Vol. CEE DP 47. Bořík, Vladimír and Caban, Matúš. 2013. Pilotné hodnotenie dopadov vybraných opatrení aktívnej politiky trhu práce (Pilot evaluation of the impacts of selected active labour market measures). Bratislava : Ministry of Labour, Social Affair and Family of the Slovak Republic, 2013. Caliendo, Marco and Hujer, Reinhard. 2005. The Microeconometric Estimation of Treatment Effects - An Overview. IZA Discussion Paper No. 1653. July2005, 2005, Vol. 1653. European Commission. 2013. Apprenticeship and Traineeship Schemes in EU27: Key Success Factors. s.l. : European Union, 2013. Heckman, James J., Ichimura, Hidehiko and Todd, Petra E. 1997. Matching as an econometric evaluation estimator: Evidence from evaluating a job training program. Review of Economic Studies. 1997, Vol. 64, pp. 605-654. Juznik Rotar, Laura. 2012. How Effective is the Slovenian institutional Training Program in Improving Youth's Chances of Reemployment? Eastern European Economics. 2012, Vol. 50, 3, pp. 94-106. Lechner, Michael and Melly, Blaise. 2007. Earnings Effects of Training Programs. IZA Discussion paper. 2007, Vol. Discussion Paper No. 2926. http://ftp.iza.org/dp2926.pdf

Author Information

Miroslav Štefánik (presenting / submitting)

Slovak Academy of Sciences

Institute of Economic Research

Bratislava

Search the ECER Programme

Search for keywords and phrases in "Text Search"
Restrict in which part of the abstracts to search in "Where to search"
Search for authors and in the respective field.
For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.