Synthetic Control Methods (SCM) with Machine Learning (ML) and Traditional Econometrics

Synthetic Control Methods (SCM) with Machine Learning (ML) and traditional econometrics share the goal of causal inference, but they differ fundamentally in their approach, assumptions, tools, and applications. Here’s a breakdown of their key differences:

---

1. Core Philosophy & Methodology

Traditional Econometrics	SCM + ML
Focuses on parametric models (e.g., linear regression, IV, DID) with strict assumptions (linearity, exogeneity).	Uses non-parametric or semi-parametric ML models (e.g., causal forests, neural nets) to relax rigid assumptions.
Relies on statistical theory (e.g., OLS, maximum likelihood) for inference.	Combines algorithmic optimization (e.g., gradient descent, regularization) with causal frameworks.
Requires manual model specification (e.g., choosing control variables).	Automates feature selection and model tuning (e.g., LASSO for variable selection).

Example:

• Traditional: Estimating a policy’s effect using Difference-in-Differences (DID) with fixed effects.
• SCM+ML: Creating a “synthetic” counterfactual using ML-weighted donor units and high-dimensional data (e.g., satellite imagery).

2. Handling of Data

Traditional Econometrics	SCM + ML
Works best with structured, low-dimensional data (e.g., GDP, employment rates).	Excels with unstructured/high-dimensional data (e.g., text, images, sensor data).
Limited ability to process big data (e.g., millions of observations).	Built for scalability using ML pipelines (e.g., TensorFlow for feature extraction).
Relies on pre-specified relationships (e.g., linearity between variables).	Discovers non-linear patterns and interactions (e.g., how social media sentiment affects consumer spending).

Example:

• Traditional: Using panel data regression to study the impact of education spending on GDP.
• SCM+ML: Analyzing satellite night-light data + Twitter sentiment to create synthetic controls for regional development policies.

3. Assumptions & Flexibility

Traditional Econometrics	SCM + ML
Requires strong identifying assumptions (e.g., parallel trends in DID, exclusion restriction in IV).	Often relaxes assumptions by leveraging data richness (e.g., ML identifies latent confounders).
Vulnerable to model misspecification (e.g., omitting a key variable).	Reduces misspecification risk via automated feature engineering.
Struggles with heterogeneous treatment effects (e.g., “one-size-fits-all” estimates).	Explicitly models heterogeneity (e.g., causal forests to estimate effects for subgroups).

Example:

• Traditional: Assuming all regions respond equally to a tax cut (average treatment effect).
• SCM+ML: Identifying that urban areas benefit more than rural ones from the same policy.

4. Interpretability vs. Predictive Power

Traditional Econometrics	SCM + ML
Prioritizes transparent, interpretable models (e.g., regression coefficients show marginal effects).	Often trades interpretability for accuracy (e.g., neural nets as “black boxes”).
Results are theory-driven (e.g., testing hypotheses derived from economic models).	Results are data-driven (e.g., patterns emerge from the data without prior theory).
Uncertainty quantification via confidence intervals/p-values.	Uses Bayesian ML or bootstrapping for uncertainty (less standardized).

Example:

• Traditional: A regression coefficient showing a 2% GDP boost per 1% education spending increase.
• SCM+ML: A synthetic control model predicting that Policy X saved $10M in healthcare costs, but the “why” is less clear.

5. Use Cases

Traditional Econometrics	SCM + ML
Policy evaluation with clean, small-N data (e.g., minimum wage laws).	Complex interventions (e.g., COVID-19 lockdowns) with messy, large-N data.
Academic research (e.g., testing economic theories).	Industry applications (e.g., Uber using SCM+ML to measure driver incentive impacts).
Macroeconomic forecasting (e.g., ARIMA models for inflation).	Micro-level causal inference (e.g., personalized marketing using customer-level data).

6. Tools & Workflow

Traditional Econometrics	SCM + ML
Software: Stata, EViews, R (lm, plm).	Tools: Python (EconML, DoWhy), R (tidysynth).
Workflow: Hypothesis → Model → Test.	Workflow: Data → Algorithm → Validation → Iterate.

---

Key Takeaway

SCM + ML does not replace traditional econometrics but augments it by:

1. Solving problems where classical methods fail (e.g., high-dimensional data).
2. Automating tedious tasks (e.g., donor pool selection in SCM).
3. Enabling causal inference in real-world, messy settings (e.g., tech A/B testing).

However, traditional econometrics remains vital for theory testing, transparency, and policy debates where interpretability trumps predictive power.

When to Use Which?

• Traditional Econometrics: Small datasets, theory testing, regulatory compliance (e.g., antitrust analysis).
• SCM + ML: Big data, real-time decision-making, heterogeneous effects (e.g., personalized policies).

Let’s dive into a real-world case study comparing traditional econometrics and SCM + ML in action. We’ll use the classic example of evaluating the impact of California’s 1988 tobacco control program (Proposition 99) on cigarette sales. This example highlights how the two approaches differ in methodology, assumptions, and results.

---

Case Study: California’s Anti-Smoking Policy (Prop 99)

Intervention: California implemented a large-scale tobacco control program in 1988 (higher cigarette taxes, advertising bans, public health campaigns).
Goal: Estimate the causal effect of Prop 99 on per-capita cigarette sales.

1. Traditional Econometrics Approach (Difference-in-Differences – DID)

Methodology:

• Control Group: Compare California to other U.S. states without similar policies.
• Model:
• Cigarette Salesst = α + β⋅CAs⋅Postt + γXst + ϵst​
  • CAs⋅Postt​: Interaction term (treatment effect).
  • Xst​: Controls like state income, population.

Results:

• DID Estimate: Prop 99 reduced cigarette sales by ~20 packs per capita.
• Limitations:
  • Parallel Trends Assumption: Assumes pre-1998 trends in cigarette sales for California and control states would have remained parallel without Prop 99 (hard to verify).
  • Omitted Variables: Fails to account for state-specific unobservables (e.g., cultural shifts against smoking).
  • Simplistic Control Group: Uses all non-CA states, including those dissimilar to California (e.g., Nevada with tourism-driven sales).

2. SCM + ML Approach

Methodology:

• Synthetic Control: Construct a “synthetic California” as a weighted combination of donor states that closely matches California’s pre-intervention cigarette sales trends and covariates.
• ML Enhancements:
  1. LASSO Regression: Automatically select relevant donor states (e.g., Colorado, Utah) from a large pool.
  2. Regularization: Optimize weights to avoid overfitting.
  3. High-Dimensional Data: Incorporate additional predictors (e.g., state-level smoking surveys, tobacco lobby spending).

Results:

• SCM + ML Estimate: Prop 99 reduced sales by ~27 packs per capita.
• Key Advantages:
  • Flexible Donor Pool: ML selects states that mirror California’s pre-1988 trends (e.g., Utah for low smoking rates, New York for urban population).
  • Robustness Checks: Quantify uncertainty via placebo tests (e.g., applying SCM to untreated states).
  • Heterogeneous Effects: ML reveals the policy worked better in urban vs. rural counties (uncovered via causal forests).

Side-by-Side Comparison

Aspect	Traditional DID	SCM + ML
Control Group	All non-CA states (including poor matches).	ML-selected donors (e.g., CO, UT, NY).
Assumptions	Parallel trends (untestable post-treatment).	Matches pre-treatment trends exactly.
Data Used	Basic covariates (sales, income).	High-dimension data (surveys, lobby spending).
Treatment Effect	-20 packs per capita.	-27 packs per capita (larger, more precise).
Interpretability	Simple coefficient (average effect).	Granular insights (urban/rural differences).

Why SCM + ML Outperformed Traditional Econometrics

1. Better Pre-Intervention Fit: SCM + ML synthetically replicates California’s pre-policy trends, while DID relies on weaker assumptions.
2. Reduced Bias: ML discards poor donor states (e.g., Nevada) that distort DID estimates.
3. Richer Insights: Heterogeneous effects guide targeted policy adjustments.

Key Takeaways

• Traditional Econometrics works for simple, theory-driven analyses with clean data.
• SCM + ML shines in complex, real-world settings with messy data and unverifiable assumptions.
• Hybrid Approaches (e.g., SCM + DID) are now emerging to combine strengths of both.

This article is written with the help of deepseek.ai

Reference: Prop 99 case study, cite Abadie et al.’s paper:
Abadie, A., Diamond, A., & Hainmueller, J. (2010). Synthetic control methods for comparative case studies: Estimating the effect of California’s tobacco control program. Journal of the American Statistical Association.