By Patrick Arni, Gerard van den Berg, and Rafael Lalive
The standard empirical evaluations of labor market policy only consider the direct effects of single programs on their participants. This column* (based on IZA Discussion Paper No. 9457) argues that this fails to capture important aspects of real-world labor market policy – policy regimes and strategies. Using Swiss data, it employs a novel empirical approach that concurrently examines the effects of supportive and punitive policies (‘carrots’ and ‘sticks’). Policy regimes are shown to exert economically relevant effects, and accounting for these effects is crucial when designing labour market policy.
In many countries, governments spend substantial amounts on measures of labor market policy (LMP) for unemployed workers with the aim of increasing their chances and speed of finding a job. Spending on LMP may go above 1% of GDP in some OECD countries, reaching maximum burdens during recessions. During the Great Recession in the US, the OECD strongly recommended spending additional resources on supportive LMP, notably training and job search assistance. Surprisingly, empirical evidence on the effectiveness of LMP as a system is broadly missing, despite its widespread use in many OECD countries.
In the last two decades a considerable empirical literature on the effects of single LMP measures – like training, job search assistance, or sanctioning – has emerged (see IZA Discussion Paper No. 9236 for an overview). But in the real-world context of current unemployment insurance (UI) systems, several LMP programs run at once, following the policy strategies defined by public employment service (PES) authorities and implemented by caseworkers with discretionary powers. Depending on the goals of (local) policymakers, the weights on different types of programs and ‘philosophies’ may differ – do they provide support or put pressure on job seekers?
Assessing policy regimes
Getting a consistent picture on the effects of LMP within a UI or welfare system requires more than evaluating the direct treatment effects of a single program. We focus on one key aspect: the regime effect of a program. We operationalize this by how intensively a caseworker or a PES intends to use a program. Policymakers and implementers, PES agencies and caseworkers produce policy regimes which act over and beyond single treatments.
Regime effects may act through the general comprehensive guidance approach of the PES agencies and the caseworkers towards their clients, for example, through applying different levels of strictness or different frequencies of use of certain policy types. Moreover, regimes may operate via information spillovers. Strongly supportive regimes may invest more effort in getting to know the job seeker in order to set up a more targeted treatment plan.
A more effective service may enhance the job seeker’s search efficiency (how and where to search) and increase job offer arrival rates, also before participation or even without individuals actually joining an LMP program. The same applies to restrictive policies like sanctioning and monitoring; they may affect job seekers through ‘threat’ or pressure, also without the sanction being actually (yet) imposed. Thus, the potential relevance of regime effects is supported by the fact that they can affect everyone in the system, treated or untreated.1
We show how regime effects can be identified and estimated using detailed large-scale register data on job seekers in Switzerland. We discuss how to measure policy regime intensities and develop a framework to jointly estimate different effects – of regimes by PES agencies and caseworkers on the one hand, and of specific treatments on program participants on the other hand.
Moreover, we distinguish the effects of supportive policies (‘carrots’) – notably, training and job search assistance programs – from the effects of punitive policies (‘sticks’) – sanctions and workfare programs. Assessing all these effects in one model offers the crucial advantage that we can evaluate the relative importance of different elements in the system as well as their interplay. This offers a substantially more comprehensive picture of the impacts that labor market policy really exerts on concerned individuals.
Results: Regime effects matter
We exploit detailed register data for 130,000 individual job search spells for men of all ages in Switzerland between 2000 and 2005. Switzerland is particularly suitable for such an exercise since the different LMP intensities vary substantially across the country. We combine registers that report LMP events in daily precision, detailed socio-demographic and UI benefit-related eligibility information as well as individual social security earnings data for several years before and after unemployment. Importantly, we observe policy implementers – PES agencies and caseworkers – in the data as well.
The study considers two individual labor market outcomes, which cover the mid run and the short run: average monthly earnings within the 3.5 years after unemployment exit on the one hand, and unemployment duration on the other. Figure 1 visualizes some key results on the effects of different types of regimes and treatments on mid-run earnings. As to be expected from existing evidence, being treated by at least one ‘stick’ significantly reduces individual earnings after unemployment; on the other hand, participating in a supportive program affects earnings positively.
Figure 1. Treatment and regime effects of Carrots and Sticks, in % of average earnings
Notes: Effects are expressed in percent of average monthly earnings within 3.5 years after unemployment (3547 CHF = 3290 EUR = 3575 USD in sample). Treatment effects: effects of being exposed to at least one carrot (job search assistance, training) or stick (sanction, workfare program). Regime effects: marginal effect of changing policy intensity by 0.1.
Figure 1. Treatment and regime effects of Carrots and Sticks, in % of average earnings
However, on top of these effects directly arising from program participation, we find economically relevant effects of supportive and punitive policy regimes. Both PES agencies and caseworkers contribute to these additional LMP effects. Increasing the intended frequency of use of restrictive policies by 10 percentage points (i.e., about half a standard deviation) reduces realized earnings by about 4%. Increasing the use of supportive policies, on the other side, improves earnings by about 2.5%.
Interestingly, more intense use of both carrot and stick regimes causes significant reductions in unemployment duration. It is important to note that these regime effects apply to everyone at any time, not only to program participants after treatment as in the case of the ‘classical’ treatment effects. This underlines the economic relevance of these newly identified LMP effects.
We also address the question how labor market policy mixes could be optimized. Our study provides a set of ‘experiments’ which simulate how marginal policy regime changes would affect earnings and unemployment durations of the job seekers as well as the cost of UI. Figure 2 summarizes the results for two ‘experiments’. The first assumes that PES agencies and caseworkers would decide to increase the use of both types of regimes – carrots and sticks – by half a standard deviation (about 10 percentage points) above median policy intensity. The second assumes an increase in the supportive policy regime and a decrease in the punitive regime of the same size.
Figure 2. Marginal effects of changing policy regimes: Two scenarios
Notes: Marginal effects of changing the regimes by 0.5 standard deviation (i.e., about 0.1 in policy intensity). Earnings changes: in percent of average monthly earnings of non-treated individuals. Cost changes: in percent of total benefit cost per person.
The results are striking in documenting the relevance of policy regimes and their interplay. Intensifying both carrots and sticks generates a relevant reduction in individual unemployment duration (-20 days), at the expense of achieving lower earnings (-3%) after unemployment. UI saves about 6.2% of cost, mostly due to the reduced unemployment durations. The second experiment, which puts more weight on supportive regimes, is favorable for the individual’s pay, at the expense of some additional cost for UI.
Conclusion
This novel empirical approach of jointly assessing the effects of labor market policies demonstrates that we miss out on an important part of LMP impacts when only focusing on the consequences of single treatments. Policy regimes matter – their effect, which comes on top of classical program effects, is significant. Considering regime effects can change the assessment of the total effect of certain types of policies on individual labor market outcomes. For instance, supportive policies are positive when taking into account regime effects. Only considering short-run treatment effect results in a negative evaluation, since unemployment duration appears to be prolonged (due to the so called ‘lock-in’ effect).
However, when including regime effects the picture changes – supportive regimes as a whole can reduce unemployment durations and show positive impacts on individual earnings. Applying a comprehensive framework to UI or welfare registers in different countries can offer instructive new empirical tools for policy optimization.2 The framework can easily be extended to other different types of assessed policies. Applying it on a large scale, covering different policy intensities, can produce useful sets of estimates of marginal effects of LMP. These can be used as inputs for policy optimization exercises. Policymakers need, however, to first fix their objectives. What is most important – reducing unemployment durations, improving individual job stability and welfare, or saving on the UI budget?
(*This column was originally published on voxeu.org; edited and reposted with permission.)