IZA Newsroom

IZA – Institute of Labor Economics

Mark Fallak

Major replication project re-analyzes over 100 economics and political science articles

by

Replication and reproduction of research findings are crucial for scientific progress. These processes allow the scientific community to assess the reliability of research outcomes, fostering a self-correcting mechanism and informing policy decisions.

A new IZA discussion paper by Abel Brodeur and over 350 coauthors investigates the reproducibility of findings in leading economics and political science journals (2022-onwards). The study computationally reproduces and conducts sensitivity analyses for 110 articles. Small teams of “replicators” – PhD students, postdoctoral fellows, faculty, and researchers with PhDs – collaborated on analyzing individual studies. Each team computationally reproduced the results, identified coding errors, conducted sensitivity analysis, and wrote a report shared with the original authors.

High rate of reproducible results, but many coding errors

The study finds a high rate (over 85%) of fully computationally reproducible results. This means the provided code by the original authors can be run and produce the same numerical results reported in their articles. This is likely due to data and code availability policies enforced by the journals, which often include a data editor.

However, having fully reproducible results does not guarantee the absence of coding errors. Excluding minor issues, the study identified coding errors in about 25% of studies. While not all errors affect the final conclusions, some major ones were found, including a large number of duplicated observations, incomplete interaction in difference-in-differences regressions, miscoding of the treatment variable, and model misspecification.

Are the results robust?

Replicators conducted 5,511 re-analyses, involving changes to the weighting scheme, control variables, estimation methods, or using new data.

A visual analysis of the test statistics distribution (see figure below) shows a large shift in the mass of test statistics from the just statistically significant at the 5% level region to the statistically insignificant and 10% significance regions after re-analysis. This suggests re-analyses decrease the statistical significance of many originally published test statistics.

In terms of relative effect size, most re-analyses cluster around one, with about 17% being smaller or equal to 0.5. Around half report a ratio greater than one, suggesting potential conservatism from the original authors.

The study defines robustness reproducibility as having the same sign and remaining significant at the 5% level after re-analyses. Using this definition, a robustness reproducibility of about 70% is found. Additionally, half of original point estimates significant at the 10% level (but insignificant at the 5% level) become statistically insignificant at the 10% threshold with the replicators’ robustness checks. For original estimates significant at the 5% level (but insignificant at the 1% level), over a quarter of re-analyses become insignificant at the 10% threshold.

Which robustness checks matter more?

The study’s robustness checks can be broadly categorized into eight groups. Robustness reproducibility rates were found to be lower when replicators changed the dependent variable (45%) and the sample (64%), while introducing new data yielded the highest rates (87%). The remaining categories, including changing control variables, estimation methods, inference methods, main independent variable, or weighting scheme, offered robustness rates around 75%.

Barriers to reproducibility

The lack of raw data significantly restricted replicators’ abilities across all analysis categories. Raw data limitations hindered robustness checks for 19% of teams and re-coding key variables for 18%. Additionally, 12% and 13% of teams believed the lack of raw data impeded their ability to perform replications and extensions, respectively. Furthermore, 7% of teams felt the original paper was unclear to the point of hindering robustness checks.

Conclusion

The large scale of this ongoing project has the potential to influence research norms and researcher behavior by promoting more rigorous methodologies and discouraging questionable research practices.

On a positive note, over 40% of replicators reported a more optimistic view of the discipline due to the quality of the replication packages they reproduced.

Men face a tougher time finding part-time work

by

Many juggle work and family commitments, making part-time jobs a desirable option. A recent IZA discussion paper by Daniel Kopp delves into this topic, exploring how readily men and women find part-time work in Switzerland.

Using large-scale data from an online recruitment platform and an online job board, Kopp’s analysis exposes a concerning trend: recruiters favor full-time applicants over part-time ones. Interestingly, this “part-time penalty” is harsher for men than women.

Even when accounting for job and workplace characteristics, the bias persists. The study suggests gender stereotypes as a significant factor behind this disparity. Men seeking part-time work face greater resistance compared to women, potentially hindering a more balanced division of paid and unpaid labor between genders.

How are firms affected by parental leave absences?

by

Motherhood and parental leave are frequent causes of worker absences and employment interruptions. Paid parental leave is a key family policy helping parents to reconcile work and family life, but it presents logistical challenges to firms in managing these employment gaps. The potential costs are frequently raised as arguments against more generous parental leave policies.

In a new IZA discussion paper, Mathias Huebener, Jonas Jessen, Daniel Kuehnle and Michael Oberfichtner study how small- and medium-sized firms are affected by parental leave absences using linked employer-employee data from Germany.

Increased pre-leave hiring, incomplete replacement

Analyzing high-frequency data, the study shows a rise in firm hiring around six months before childbirth. However, this increase suggests replacement for only one-third of mothers. Notably, replacement hiring is significantly higher for mothers with fewer colleagues performing similar tasks, which implies they cannot easily be replaced internally.

Most pre-leave hires have similar demographic characteristics as mothers (young women). Interestingly, there is no evidence that they are more likely to exit the firms again upon the mothers’ return, suggesting that joining a firm as a parental leave replacement can be an important stepping stone into permanent employment.

No long-term impact of extended leave

The study further examines the effects of a 2007 leave extension for higher-earning mothers. While the reform delayed mothers’ return to their pre-birth firms in the first year, it had no lasting impact on their long-run employment outcomes. The delay was more significant for mothers with fewer co-workers, who previously took shorter leave.

During the extended parental leave period, firms showed a reduction in employment levels, suggesting that replacement hiring did not fully cover the absences. However, there were no effects on firms’ employment or wage bill in the long run. Firm closures rates were also unaffected by longer parental leave absences.

No reduced hiring of young women

Finally, the study investigates whether firms adjust hiring practices based on anticipated longer leave absences. Since replacements are costly, firms might avoid hiring young women who are statistically more likely to take extended leave after the reform. The well-intended policy could then backfire at the expense of young women, harming their labor market prospects.

However, comparing long-term hiring decisions in firms experiencing post-reform leave absences to those with pre-reform absences, the study finds no evidence of a negative impact on the hiring probability of young women. This holds true across various estimation methods and even for smaller workgroups where internal adjustments might be more challenging.

One explanation could be the prevalence of part-time work among mothers returning from leave in Germany. The overall impact of a longer part-time absence might be much less significant for firms than the birth-related absence itself. Additionally, because parental leave absences are known several months in advance, firms can proactively adjust processes to minimize disruption.

Work-home distances increase in Germany

by

A new IZA discussion paper by IAB researchers Sena Coskun, Wolfgang Dauth, Hermann Gartner, Michael Stops and Enzo Weber examines how the rise of remote work during the COVID-19 pandemic is impacting career choices and where people live.

The authors analyze large datasets on employment history in Germany alongside a new indicator for remote work potential across different occupations. Their findings reveal a clear trend: since the pandemic, workers in jobs conducive to remote work have begun living farther from their workplaces.

The association of working-from-home potential and work-home distance has increased significantly since 2021 as compared to a stable pattern before.

This shift is particularly pronounced for new hires, suggesting a deliberate choice towards remote jobs with greater physical distance from the office.

The research also indicates that this trend is most significant in large cities, suggesting that remote work offers an escape from tight housing markets. Interestingly, the study doesn’t find a difference in how this trend affects men versus women.

First-generation high achievers face hidden obstacles on path to success

by

High-achieving university graduates from families without a history of higher education, despite their academic success, encounter disadvantages in their academic and professional journeys, according to an IZA discussion paper by Nikki Shure and Larissa Zierow.

The researchers examined data on scholarship recipients in Germany, all ranking in the top percentile nationally. Surprisingly, even these top performers from working-class backgrounds were more likely to choose universities closer to home and with lower prestige compared to their peers with college-educated parents.

The study suggests these first-generation graduates may prioritize factors beyond academic ranking, such as staying close to family or minimizing costs. Additionally, they were less likely to pursue fields with high earning potential and gravitated towards jobs with greater stability. The research also highlights a gender gap, with first-generation female graduates appearing to undervalue the professional networking opportunities offered by the scholarship program.

These findings point to hidden obstacles faced by high-achieving students from working-class families, even after gaining entry into top academic programs. Further efforts may be needed to ensure all students, regardless of background, can leverage their talents to achieve their full potential in the job market.

Vocational language courses boost skills but struggle to directly improve job prospects

by

Since 2016, the German government has been offering Berufssprachkurse (BSK), or vocational language courses, to support the labor market integration of immigrants with a need for German language training. By the end of 2022, 775,000 people had started a BSK course.

A research group commissioned by the Federal Ministry of Labor and Social Affairs to investigate the effects of BSK found that participants significantly improved their German language skills compared to similar non-participants. However, the study found former BSK participants were less likely to be employed within 30 months of starting the course. This implies that employment rates remain lower even after completion of the course, which is usually full-time.

The study does highlight positive impacts. In particular, BSK graduates were more likely to pursue further training or education compared to the control group. Whether this translates into improved employment outcomes in the long run remains to be investigated.

The report proposes several improvements to BSK, including:

  • Earlier career guidance: Job centers and employment agencies should offer career advice to BSK participants before they finish the course.
  • More part-time options: Expanding part-time BSK courses, including online or hybrid formats, would allow more working individuals to improve their German.
  • Better childcare support: The government should explore ways to help BSK participants with childcare needs.
  • Improved teacher conditions: Better pay, more teaching autonomy, and flexible work models could attract and retain qualified BSK instructors.

IZA study analyzes HR practices to recruit and retain skilled workers

by

German companies are struggling to hold onto skilled workers, according to a new IZA Research Report commissioned by the Federal Ministry of Labor and Social Affairs. The researchers analyzed data from the Linked Personnel Panel (LPP), a nationwide survey of businesses and employees over an eight-year period. The evaluation of the survey data collected every two years from 2012 to 2020 underlines the growing challenge of securing skilled workers, but also shows opportunities and key areas for corporate human resources to address this challenge.

The findings reveal a concerning trend: the number of applicants per open position is dropping, it takes longer to fill vacancies, and employee turnover intentions are rising. While companies acknowledge the importance of a positive work environment and competitive pay in retaining staff, the study identified a troubling gap between this awareness and actual implementation.

Despite recognizing the benefits of a good company culture, many businesses haven’t translated this knowledge into actionable practices. Employees report a decline in overall work atmosphere, suggesting companies are failing to address their concerns. The report highlights the need for companies to bridge this “knowing-doing gap” by actively implementing strategies that promote employee satisfaction and loyalty.

25th IZA Summer School: Applications open!

by

Fostering the next generation of researchers is a cornerstone of IZA’s mission. The IZA Summer School provides an unparalleled opportunity for PhD students to gain insights from leading international researchers on cutting-edge advancements in their fields. It’s a vibrant platform for discussing research, interacting with faculty, and building relationships with fellow students.

In celebration of its 25th anniversary (July 15-19, 2024), the IZA Summer School will embark on a new chapter. The program will now alternate its main focus between labor economics and behavioral economics each year.

This year’s group of lecturers include highly renowned international economists working at the intersection of both fields:

  • Stefano DellaVigna is a Professor of Economics and Business Administration at UC Berkeley and Co-Director of the Initiative in Behavioral Economics and Finance. He is also a co-editor of the American Economic Review.
  • Ernst Fehr is a Professor of Economics at the University of Zurich. He is a pioneer in behavioral economics and co-founder of the Zurich Center for Neuroeconomics.
  • Ulrike Malmendier is a Professor of Economics and Finance at UC Berkeley and Co-Director of the Initiative for Behavioral Economics and Finance. She is also a Member of the German Council of Economic Experts.
  • Andrew Oswald is a Professor of Economics and Behavioural Science at the University of Warwick. He is also an honorary Senior Research Fellow in the Wellbeing Research Centre at Manchester Harris College, University of Oxford, and Chair of the IZA Network Advisory Panel.
  • Amelie Schiprowski is an Assistant Professor of Economics at the University of Bonn and a faculty member of the ECONtribute Cluster of Excellence. She is also an Alumna of both IZA and the IZA Summer School.
  • Florian Zimmermann is a Professor of Economics at the University of Bonn, member of ECONtribute and IZA Research Director, having served in this capacity at the briq Institute of Behavior & Inequality before its reintegration into IZA.

+++

Applications for the 25th IZA Summer School are now open until March 31, 2024.

Award-winning research in the economics of climate change

by

To foster research into the nature and implications of climate change, IZA gives an award for “Innovative Research in the Economics of Climate Change” (IRECC) for the two best topical IZA Discussion Papers of the previous year. Worth 10,000 euros, the IRECC Award recognizes important new insights into the broader, often underestimated consequences of climate change and the effects of environmental policies on society and the labor market.

Real-world willingness to pay for carbon offsets

One of the two papers selected for the 2024 IRECC Award is “Willingness to Pay for Carbon Mitigation: Field Evidence from the Market for Carbon Offsets” (IZA DP No. 15939). In this study, Matthias Rodemeier sheds light on a crucial question: how much are people truly willing to pay to protect the environment? He does so by analyzing real-world behavior rather than hypothetical surveys.

Rodemeier examines the choices of over 250,000 German delivery service customers that were offered voluntary carbon offsets. Interestingly, consumer demand for offsets increased when prices were subsidized but not when the compensated amount of carbon was matched by the delivery service. However, transparency was key. When explicitly informed that the delivery service is matching the offsetting of emissions on its own costs, consumer behavior shifted dramatically. A salient 300% match of emissions boosted offset demand by 22%.

Thus, a simple intervention that advertises the firm’s participation in the offsetting costs makes subjects sensitive to the impact of carbon mitigation. The implied willingness to pay (WTP) for carbon mitigation increased from practically zero to €16 per tonne of CO2 (tCO2).

Two additional surveys reveal that the increase in WTP due to the firm’s contribution is mostly driven by fairness preferences and not by a higher intrinsic valuation for carbon mitigation.

This research further exposes a significant gap between what people say they would pay in surveys (hypothetical WTP) and what they actually do (revealed preferences). In this case, hypothetical WTP averaged €238/tCO2 – a staggering 1,338% higher than revealed preferences. This highlights the importance of using real-world behavior to understand true environmental values.

Long-term economic and social effects of climate change

The second award-winning paper, “The Effects of Climate Change in the Poorest Countries: Evidence from the Permanent Shrinking of Lake Chad” (IZA DP No. 16396) by Remi Jedwab, Federico Haslop, Roman Zarate, and Carlos Rodriguez Castelan, tackles a neglected aspect of climate change: its slow, gradual effects on societies. The research uses the dramatic decline of Lake Chad – once the world’s 11th largest – as a case study.

Lake Chad shrunk by 90% between 1963 and 1990 due to external factors. While water supply decreased, land supply increased, which could in theory generate both negative and positive economic effects. The researchers innovatively compiled population data for nearby regions across four African nations (Cameroon, Chad, Nigeria, and Niger) spanning from the 1940s to the 2010s.

Their findings are concerning. Population growth near the lake slowed significantly only after the shrinkage began. This implies limited ability for communities to adapt. Furthermore, the negative impacts on livelihoods – fishing, farming, and herding – outweighed any potential benefits from the newly exposed land.

The study employs a spatial model to estimate welfare losses, considering potential adaptation. The results show an overall decline of 6%, with Chad experiencing the most significant impact (9%). The model further explores the potential effects of various policies – migration, land use, trade, infrastructure, and urbanization – to understand how these factors might influence the situation.

The limited effectiveness of adaptation strategies in this case underscores the vulnerability of the poorest countries to climate change. These findings have broad implications for designing policies to support these nations in facing the challenges of a changing climate.

+++

The IRECC winners “represent the best of modern applied-economics research,” according to the award committee made up of Susana Ferreira (University of Georgia) and Andrew Oswald (IZA and University of Warwick).

How the division of roles within the family shapes job satisfaction

by

The impact of the pandemic and labor shortages has brought the issue of job satisfaction to the forefront of public discussion. Within the scientific community, this topic has been under investigation for a long time. A notable finding emerges when examining job satisfaction across genders: despite women earning less than men, they consistently report higher job satisfaction.

This observation persists even when comparing a woman and a man whose jobs have similar characteristics and are similar in other personal characteristics. The scientific literature has coined this phenomenon the “paradox of the contented female worker.”

How can the apparent satisfaction of women in lower-paying jobs be explained? In a new IZA discussion paper, Christian Bredemeier, Patrick Ndlovu, Suncica Vujić and Roland Winkler propose a novel hypothesis to explain this seeming paradox. They argue that the alignment of an employee’s preferences with the nature of their job is a pivotal factor.

Some workers may appreciate a team-oriented work environment, while others may be more interested in a more competitive work atmosphere. Consequently, workers with varying preferences may exhibit disparate satisfaction levels within the same workplace. For example, a company fostering a team-oriented work environment would experience high satisfaction among employees who prefer such an environment, while more competitively oriented colleagues would be less satisfied.

Secondary earners may choose more satisfying jobs

The study posits that women are more likely than men to choose employment opportunities that align more closely with their preferences. The authors attribute this to the typical household roles of men and women. A household’s decision-making process between opting for a better-compensated job and one that aligns with personal preferences is influenced by the relative earnings of both spouses.

When one assumes the primary responsibility for the household income, the job choice decision leans towards selecting a position with a higher salary at the expense of giving up appreciated non-pay job attributes. By contrast, individuals who take the role of secondary earner are more likely to prioritize personal preference over higher income.

Statistically, women are mostly secondary earners in their households. According to the researchers’ theory, this pattern leads to more frequent decisions of women in favor of jobs that align with personal preferences and higher job satisfaction. At the same time, this pattern leads to a strengthening of existing differences in earnings between men and women.

Job satisfaction gap among married couples with children

To test their theory, the authors analyze a combination of survey and tax data from over 5,000 Canadians, linking their information about job satisfaction with household income. The empirical findings align closely with the theory. On average, observed women report significantly higher job satisfaction than men, even after accounting for the average impact of individual and workplace characteristics.

However, this disparity in job satisfaction is not present in all population groups. It is distinctly evident within married couples with children and a traditional division of roles, where the male partner contributes more than half of the household income. In contrast, there are no discernible gender differences in job satisfaction among singles, childless couples, and couples with a less traditional division of roles. This underscores the pivotal role family decisions and within-family earner roles play in shaping an employee’s job satisfaction.