Evaluation of the Operational Programme “Employment Austria 2014 to 2020” of the European Social Fund

The accompanying evaluation of the European Social Fund (ESF) in Austria covers all thematic priorities of the programme with the exception of the new REACT-EU priority axis: employment, poverty reduction and education. Special attention is paid to the transition region of Burgenland. The cross-cutting objectives “Gender mainstreaming and equality” and “Disability mainstreaming and accessibility” are also analysed. The report contains updated analyses and assessments of the design and implementation of the programme. However, the main focus was placed on impact aspects, with the spectrum of analyses ranging from qualitative approaches to counterfactual impact analyses. Overall, the findings show the positive effects of the ESF in a variety of ways, such as the effects on the participation in education or employment integration of those supported or the development and implementation of innovative approaches. At the same time, there are also indications of challenges and potential for improvement in order to further strengthen the performance of the ESF in the future.

L&R Social Research carried out the evaluation as part of a consortium together with WIFO and IHS. L&R Social Research focussed on the measures of priority axis 2 – poverty reduction. The investment priority “Promoting the active inclusion of disadvantaged groups and groups at risk of poverty” represents a quantitatively central element of the ESF programme 2014-2020 with around 31% of the funding (excluding REACT-EU). IP 2.1 is also a quantitatively significant priority in terms of the number of participants: according to the ESF monitoring system ATMOS, 90,639 people had taken part in measures in this IP by the end of 2020, which is 42% of all participants in the ESF programme.

A broad methodological approach was chosen to analyse implementation and impact. Based on the analysis of the monitoring data and existing studies and evaluation reports, various surveys were carried out as part of priority axis 2: Two online surveys of project managers and in-depth interviews with them and the intermediary bodies involved were conducted. This formed an important basis for the impact analysis, which is based on several pillars: In addition to outcome indicators according to ESF monitoring, a comprehensive counterfactual impact analysis was carried out.

As part of the counterfactual impact analysis, the labour market policy impact of participation in stabilisation measures or target group-oriented employment projects on the people receiving support was presented. The central question here is what effect on medium and long-term labour market integration – measured in terms of periods in employment, unemployment or without labour market participation – has resulted for the individuals concerned as a result of participating in the ESF offer. The study analysed the careers from the start of the measures of people who completed the programme in 2017, focusing on participants in two types of measures that are quantitatively significant for PA 2 and for which a sufficiently long observation period is available. The data-related restrictions that had to be overcome posed a particular challenge.

The processing of the results of the counterfactual impact analysis follows a two-stage concept: in a first step, the results, also known as the “gross impact”, were presented for the beneficiaries. In the second step, a control group comparison was used after comparable individuals were identified by means of statistical matching of register data. The causal effect of participation, the so-called “net effect” of the subsidy, can be determined by comparing the results of subsidised persons and the respective comparison persons (counterfactual analysis).

The results showed that participation in stabilisation measures has a positive effect on the time spent in employment. The achievement of the goal of gradual integration into the labour market is evident from the high proportion of those receiving support who participate in further AMS labour market policy measures. Participation in employment projects increases the time spent in employment and leads to a significant reduction in the time spent in positions distant from the labour market. By participating in the project and subsequently taking advantage of further measures, important steps are taken towards integration into employment in the sense of an integration pathway.

There is also no significant difference in the development of earned income between the employment projects and the control group. However, due to the fact that the difference between the study and control group decreases over time, a positive income balance can be expected for the sixth or seventh subsequent year if this trend continues.

In the case of employment projects, a strong net increase in employment periods of around 64 days can already be observed in the first year after joining as a transit worker.

The report is available here.