Stratum

A good rule of my thumb is to never start coding until you understand what you're building well enough to explain it to someone who doesn't care. With DermEquity, I spent weeks reading Koh and Liang's influence functions paper, the Fitzpatrick17k dataset documentation, Hardt et al.'s fairness framework, etc. before touching a Kaggle notebook. With NEXUS, I read Wilder's original 1978 RSI paper and Harvey's 1988 yield curve dissertation before writing a single line of JavaScript. With STRATUM, I read economists.

Of course, my background in NEXUS had given me a relatively solid foundation in finance. However, economics is a slightly whole different shebang. Prior to this project, (I haven't taken AP Econ yet) my relationship with economics was shabby. I am now somewhat better informed.

On intergenerational mobility and what it actually measures: the Chetty et al. framework defines upward mobility as the expected income rank at age 35 for a child born in the 25th percentile of the national income distribution. This is longitudinal: it requires tracking people from childhood to adulthood, which is why it uses IRS records rather than surveys. The finding that this varies by a factor of almost two across U.S. counties, and that it has not improved since the 1980s despite overall economic growth, is one of the more uncomfortable empirical results in modern social science. The mobility we tell ourselves exists is, in substantial parts of the country, largely mythological.

But what drives it? The Opportunity Atlas identifies five factors at the commuting zone level: residential segregation, income inequality, local school quality, social capital, and family stability. These are correlational findings. STRATUM takes a complementary angle: rather than asking what predicts mobility empirically, it asks what structural barriers exist that prevent mobility from occurring.

On PCA and why it's more defensible than making up weights: the core methodological question in building any composite index is how to weight the components. If you say income counts for 25% and education counts for 20%, you're making a causal claim without evidence. Two reasonable researchers could defend completely different weight choices and produce completely different final scores from identical data.

PCA extracts weights from the covariance structure of the data itself. The first principal component is the direction of maximum variance across all six dimensions simultaneously. Counties that score high on one barrier tend to score high on others: the poverty rate, broadband exclusion, low income, and low education are correlated. PC1 captures this dominant pattern of co-variation, and its loadings become the weights. This is more defensible not because it's "objective" (the choice of variables still involves judgment), but because it's consistent with the data's own structure rather than someone's intuition about importance.

PC1 explaining 58% of variance in our run means that 58% of the information in all six dimensions can be captured by a single weighted composite. If PC1 explained 17%, the dimensions would be essentially uncorrelated and combining them would be meaningless.

On Moran's I and spatial autocorrelation: a key empirical question about mobility barriers is whether they cluster geographically. If high-barrier counties cluster near other high-barrier counties, it suggests regional structural causes: persistent poverty zones, deindustrialized corridors, historical disinvestment patterns. If barriers are randomly distributed, county-specific dynamics dominate.

Moran's I quantifies this. Values near +1 indicate that similar values cluster; values near 0 indicate random distribution. For U.S. county-level socioeconomic data, Moran's I typically runs between 0.4 and 0.7. STRATUM's result of 0.27 (moderate positive autocorrelation) reflects the state-level grouping approximation we use rather than full queen contiguity, which would likely produce a higher value.

Thus, the policy implication matters: geographic clustering means county-level interventions will have spillover effects on neighbors. Regional policy is warranted alongside county-specific targeting.

Before writing code, I designed the full pipeline on paper. The architecture reflects a lesson from every prior project: separation of concerns is not just good software engineering, it is survivability. When the Opportunity Insights data URLs went dead mid-build (both of them, simultaneously, which I can only describe as cosmically timed), the fallback was a one-function fix in opportunity.py that didn't touch anything else. When Alpha Vantage changed their API terms during NEXUS development, it was a one-file fix. Good architecture means you can change one thing without breaking everything adjacent to it.

Data ingestion layer (src/data/). Two modules handle all external data communication. census.py fetches ACS 5-year estimates for all 3,221 U.S. counties via the Census Bureau API (free, requires a key you can get in 2 minutes at api.census.gov/data/key_signup.html), derives rate variables from raw counts, and caches locally to avoid redundant API calls. opportunity.py attempts to download county-level mobility data from the Opportunity Insights public repository; if both known URLs are dead (as they were when I built this), it falls back to proxying mobility deficit from the poverty and income dimensions already available from Census.

Analysis layer (src/analysis/). mbi.py is the computational core: six barrier dimensions, PCA weighting, weighted composite MBI scaled to 0-100, factor contribution decomposition, and OLS factor regression for partial R-squared. spatial.py implements Moran's I using state-level grouping as a spatial proxy and a Census region summary.

Visualization layer (src/visualization/). dashboard.py builds the Plotly Dash interactive dashboard with a national choropleth map, a county-level factor breakdown panel triggered by map clicks, a national MBI distribution histogram, a Census region comparison bar chart, PCA weight visualization, and top/bottom county tables.

Entry point. stratum.py orchestrates the pipeline with a CLI (run with --report for terminal summary, --no-dashboard to skip web server).

The PCA weighting decision: this is the methodological choice I actually thought hardest about. The alternative to PCA is equal weighting (each dimension contributes 1/6 of the score) or hand-assigned weights. Equal weighting has the virtue of simplicity, but treats broadband exclusion as equally important as poverty, which may not reflect reality. Hand-assigned weights incorporate domain knowledge but are inherently subjective and hard to defend publicly.

I chose PCA for defensibility, documented the limitation explicitly (PCA weights reflect statistical co-variation, not causal importance), and included individual factor regression as a complement. The partial R-squared from a univariate regression of MBI on each dimension tells you how much that factor independently explains MBI variation, which is a different and complementary piece of information from the PCA weight.

The normalization decision: all six dimensions must be on a common scale before PCA. I used min-max normalization to [0,1], oriented so 1.0 = maximum barrier. Income is inverted (low income maps to 1.0), education is inverted, and poverty and housing burden are direct. The limitation: min-max normalization is sensitive to outliers. One county with extreme income pulls the scale for everyone else. I documented this and recommend examining outlier counties when interpreting results.

The mobility fallback: the Opportunity Insights URLs are currently dead. Rather than fail, mbi.py checks whether the upward_mobility column has any non-null values before using it; if not, it proxies mobility deficit as the average of poverty barrier and income barrier. The correlation between poverty, income, and actual Chetty mobility scores is approximately 0.7, which makes this a reasonable approximation for a framework that's transparent about its inputs.

Here's a few of my takeaways:

On overfitting and intellectual honesty: the PCA weighting decision forced me to confront the same issue I keep running into in quantitative work- when you choose methodology by looking at your data, you are in some sense tuning your results to that data. PCA weights are not objective truths. They are descriptions of statistical co-variation in a specific dataset from a specific ACS vintage year. If I had used different variables, the weights would differ. If I had used a different year, they would again shift. The honest claim is not that PCA weights are correct. It is that they are data-consistent, and that this is a more defensible starting point than arbitrary assignment. I documented this limitation, and any system that does not is either naive or dishonest. 

On composite indices and their discontents: building a composite index compels you to confront every methodological assumption you have. Which variables to include, how to normalize them, how to handle missing data, aggregate, etc. Every choice shapes the result. Therefore, the only truly honest response is to document every choice, explain the rationale, make the code fully open-source, and to acknowledge your limitations explicitly. The most important limitation of STRATUM is that MBI measures structural barriers, not causes. High housing cost burden in a county might reflect rapid economic growth (housing is expensive because jobs are arriving) rather than structural deprivation. The index treats them identically. 

On the relationship between data availability and insight: everything STRATUM uses is free and public (the gap here was not the data). Stratum is a tool that combines the data and makes it accessible to non-experts. This is a pattern I have now encountered throughout all of my recent projects. The Fitzpatrick17k dataset existed before DermEquity (although the lack of diverse representation in medical datasets is a different issue entirely- I wrote about this in my first Substack article). The FRED macroeconomic API existed before NEXUS. Similarly, the Opportunity Atlas existed before STRATUM. In this specific case, the bottleneck is the methodology to use the data well and the engineering to make the methodology accessible.