↵ select ↓ ↑ navigate esc close

marginal revolution read out: Experimentally validated digital twins

marginal revolution read out ·

Listen: https://serve.buildtall.systems/ef8bd0385d0cc8af8d13e4a260260b15b1f26cf30d0b8a9db8235e64f221f788.mp3

Source: https://marginalrevolution.com/marginalrevolution/2026/11/experimentally-validated-digital-twins.html?utm_source=rss&utm_medium=rss&utm_campaign=experimentally-validated-digital-twins


This is Marginal Revolution Read Out, and today's post is Experimentally validated digital twins. The post covers one paper, a new study by Raj Chetty, Jamie Fogel, Lawrence F. Katz, Kadeem Noray, Sonya Porter, and James Reisinger, titled Experimentally validated digital twins. The author's take is that work of this sort is going to revolutionize economic method.

The paper revisits a long-standing problem in program evaluation. For years, the field has prioritized evidence from randomized controlled trials, or RCTs, because earlier research found that observational estimates of program impacts often diverged substantially from experimental benchmarks. That earlier finding, though, was based on small survey datasets. The new paper asks what happens in the era of large-scale administrative data.

Using anonymized income tax records covering all American adults between 1999 and 2025, the researchers estimate the effects of workforce training programs on earnings. They do this by using machine learning models to predict counterfactuals from rich data on prior work histories, demographics, and residential locations. Rather than testing whether their observational estimates are unbiased, they use 37 RCTs to estimate the distribution of their bias. The result is striking: their machine learning estimates closely match the RCT estimates, with root mean squared error comparable to that of the RCTs themselves. Their method reduces root mean squared error by 60 percent relative to parametric estimators used in prior work, because larger samples and richer covariates permit more flexible, higher-dimensional models of the counterfactual.

Applying the method to a broader set of sites and enrollment years beyond the experimental sample, the authors find that treatment effects on earnings vary substantially across locations and over time, even within a given program. Programs that place more participants into target-sector jobs generate the largest earnings gains, and those impacts persist even after participants leave their initial jobs. The paper argues these findings provide guidance for improving job training programs and, more broadly, illustrate a scalable approach to program evaluation that uses observational data to construct experimentally validated digital twins.

The significance here is methodological. If observational data can be made to match experimental benchmarks this closely, then the expensive and often narrow reach of randomized trials may be supplemented, or in some cases replaced, by well-built counterfactual models. That is why the author says this kind of work is going to revolutionize economic method. The episode's single link is a dense but important one, and the claim is not that observational methods are now perfect, but that their bias can be measured and, in this setting, is small enough to be useful.

  1. Experimentally validated digital twins