Transparency and Reproducibility in Policy Analysis

The goal is to uncover completely the process of specific policy analyses, and transform them into living documents that anybody can download, reproduce and critically appraise with the least amount of barriers possible. This way researchers and analysts can have a constructive debate on the specifics of how the final output is generated and updated. Following the practices of open source software, anybody can suggest improved versions of the analysis, and all changes are observed and tracked openly.

Policy #1: Minimum Wage in the US

What is the policy?

Raising the federal minimum wage from $7.25 to $10.10 and indexing it to inflation.

What are the quantifiable gains and losses?

Raising the minimum wage would mainly generate one type of winners and two types of losers. The winners would be families with workers whose wages increase because of the policy. The first type of losers would be the families that have to pay for such increment in wages (either through reduced business profits or increase in prices). The second type of looser would be the families with workers that loose their jobs because of the new policy. The distribution of all these effects is also an objective and quantifiable dimension.

What is the most objective and transparent policy report up to date?

“The Effects of a Minimum-Wage Increase on Employment and Family Income”, 2014.

Can we make a version that is fully transparent and reproducible?

Yes. Here is a dynamic document that does that, and here is the source code that generate such document.

What did we learn by increasing the TR?

  • From a policy perspective, the academic debate around the effects on employment seems largely inconsequential.
  • There is very little knowledge around a several elements for the policy debate. The area where additional knowledge would be most consequential for the policy discussion seems to be the distribution of loses for those who have to pay for the increase in the minimum wage.

What would be the benefits of adopting this new version of the report as the official one?

  • Whenever a new report is needed, its production costs should be much smaller.
  • Before a new report is needed, the report can be improved in two dimensions:
    1- Given the current model: improve estimates in multiple parameters (distribution of losses used to pay wage increase)
    2- Revise current model: for example adding dimension over which the policy can affect the income of families (complementarity with EITC, effects over SNAP, etc)

Policy #2:?

There are many policies that I think would be interesting to put the effort to increase their TR. Here are some examples from wildly different areas:

If there is any policy that you think might benefit from this approach, please leave your suggestions in the comments below.


Leave a Comment