Causal Forge is a ground-truth causal inference laboratory. Its structural causal
model creates confounding, heterogeneous treatment effects, nonlinear outcomes, and
known counterfactuals. The benchmark measures whether estimators recover the
population average treatment effect rather than merely predicting observed outcomes.
The evaluation compares:
the unadjusted difference in observed group means;
inverse-propensity weighting (IPW);
outcome regression with separate treatment/control response surfaces;
the augmented IPW (AIPW) doubly robust estimator;
four nuisance-model regimes where the propensity model, outcome model, both, or
neither receive the correct nonlinear feature basis.
The exact potential outcomes and treatment probabilities are retained only because
this is a synthetic benchmark. They make estimator bias directly measurable.
Verified results
The benchmark ran 100 independent replications with 3,000 observations and five-fold
cross-fitting in each replication.
Nuisance-model regime
Naive MAE
IPW MAE
Outcome MAE
AIPW MAE
AIPW bias
Both correct
1.1109
0.0644
0.0388
0.0426
0.0004
Propensity misspecified
1.1109
0.1342
0.0388
0.0414
0.0008
Outcome misspecified
1.1109
0.0644
0.2688
0.0481
0.0078
Both misspecified
1.1109
0.1342
0.2688
0.3953
0.3953
This demonstrates the intended double-robustness boundary: AIPW remains accurate
when either the treatment or outcome nuisance model is correct, but not when both
are wrong. Results are Monte Carlo measurements on this synthetic SCM, not claims
about arbitrary real-world observational data.
