A Cosine Similarity of 1.000 and the Feature Still Never Fires: Causal Inertness in SAEs

description: "The 77% causal-inertness finding is now a tested, hash-verified, model-agnostic audit tool — and building it surfaced two more real bugs (a serialization leak and a three-layer reproducibility failure), a new failure-mode decomposition rooted in antipodal geometry, and features matched at cosine ≈ 1.000 that never fire once — in the good SAE."

Show me the code: the tool described here — sae-causal-audit, with its full test suite (37 tests including property-based tests), deterministic reproduction pipeline, hash-based scientific-regression gate, and a harness for auditing published production SAEs — has its own repository: mohamed-bal/sae-causal-audit. The finding that motivated it was first published alongside the original toy-model reproduction: mohamed-bal/superposition-to-monosemanticity. Every number in this piece is written by code to a results/*.json file whose SHA-256 hash is committed and verified in CI, and every figure is regenerated by re-running the corresponding script. Nothing here is hand-typed from memory.

One clarification before anything else

This piece does not claim that sparse autoencoders are broken, and it does not claim that the tool presented here certifies them. What it does is narrower and, I'd argue, more useful: it takes the causal-validation methodology from the previous piece — the one that measured, in a fully-controlled setting, that up to 77% of features passing a standard correlational recovery bar were causally inert — and turns it into an instrument anyone can point at their own SAE. An instrument, unlike a finding, has to survive contact with inputs its author never anticipated. That constraint forced design decisions, surfaced a third real bug (reported below in full, in the same tradition as the first two), and — unexpectedly — produced a refinement of the original finding itself: "causally inert" turns out to decompose into two distinct failure modes, one of which is directly traceable to the antipodal-pair geometry of superposition.

The state of the underlying question is unchanged from the previous piece: no interpretability method today gives a certified, complete account of a model's internal computation, and correlational feature-recovery metrics — the cosine-similarity matches used throughout the field's evaluation practice — measure decoder geometry, not encoder behavior. Those are two different empirical claims. This piece is about the machinery for telling them apart at whatever scale you work at, and about what that machinery revealed when it was turned, once again, on the setting where the right answers are known.

Where the last piece stopped, precisely

A one-paragraph recap, because everything below builds on it. The previous piece trained the Elhage et al. (2022) toy model — 32 sparse features compressed into an 8-dimensional bottleneck at sparsity 0.95 — then trained sparse autoencoders (L1 and TopK) on its hidden activations and asked whether the SAE's dictionary atoms recover the ground-truth features. By the standard correlational metric (unsigned cosine similarity ≥ 0.90 between a ground-truth direction and its best-matching decoder atom), recovery looked excellent for the right hyperparameters. Then came the intervention: for every matched pair, ablate the atom's code on inputs where the feature is genuinely present, and measure whether the targeted output drops more than everything else moves. For a deliberately imperfect SAE (TopK, $k=13$), 17 of 22 correlationally-matched features showed an ablation effect of exactly zero — not small, zero — because the matched atom never fired at all when its feature was presented. One of those seventeen had a cosine similarity of 0.92. The mechanism: cosine matching compares against decoder atoms ($W_{dec}$, the reconstruction vectors), while whether an atom fires is governed by the encoder and the TopK/ReLU selection — a competition the geometrically-correct atom can simply keep losing.

That was the finding. The honest criticism of the previous piece — the one I'd make of it myself — is that its most decision-relevant output was locked inside its own experiment scripts. If you trained an SAE tomorrow and wanted your inert rate, you'd have to read my code, separate the methodology from the experiment, and re-implement the former against your model. That's not a tool; that's homework. A result you can't hand to someone is a claim.

The audit, precisely

sae-causal-audit runs three measurements per matched (feature, atom) pair, in increasing order of causal strength. All three propagate through the SAE's actual encode/decode path and through a caller-supplied behavioral readout — never through decoder geometry alone.

The diagram names the split that matters: matching runs on the decoder ($W_{dec}$) because that's what correlational recovery metrics are supposed to measure, while every causal number below comes from the encoder's actual behavior through encode/decode — the two paths only merge back together in the final report.

1. Fired fraction. The cheap screen that, on its own, explained every one of the seventeen inert cases in the original experiment:

$\text{fired\_frac}_{i,j} = \frac{1}{N}\sum_{n=1}^{N} \mathbb{1}\left[\,|f_j(h_n)| > \varepsilon\,\right], \quad h_n \sim \text{feature } i \text{ present}$

If the encoder never activates atom $j$ across hundreds of samples where feature $i$ is genuinely present, no downstream claim about atom $j$ "representing" feature $i$ can be causal, whatever the cosine says. This is a screen, not a verdict — it costs one batched encode, which matters when the full ablation battery is expensive at production scale.

2. Ablation specificity. Present inputs where feature $i$ is active, encode, zero atom $j$'s code entry, decode, push both reconstructions through the downstream readout, and compare the targeted drop against mean collateral movement:

f = sae.encode(activations_on)
f_ablated = f.clone()
f_ablated[:, atom_idx] = 0.0
 
y_base = downstream(sae.decode(f))
y_abl  = downstream(sae.decode(f_ablated))
 
targeted_drop = (y_base[:, feature_idx] - y_abl[:, feature_idx]).mean()
off_target    = (y_base - y_abl).abs()[:, other_features].mean()
specificity   = targeted_drop / off_target

A causally faithful match hits its target hard and everything else barely at all. A ratio near 1 means the "intervention" is indistinguishable from generic collateral damage.

3. Sign-correct steering. The reverse direction: inputs where feature $i$ is off, atom $j$ forced to a fixed value — multiplied by the sign of the match, which the API requires as an explicit argument. Why that requirement exists is a story from the previous piece that shaped this package's interface, told properly below.

What "any SAE" means: structural typing, and why the causal path routes around the decoder

The audit accepts anything satisfying a three-member protocol — no inheritance, no registration, no adapter class to write:

@runtime_checkable
class SparseAutoencoder(Protocol):
    W_dec: torch.Tensor                                  # (d_sae, d_in), rows are atoms
    def encode(self, h: torch.Tensor) -> torch.Tensor: ...
    def decode(self, f: torch.Tensor) -> torch.Tensor: ...

sae_lens.SAE — the interface behind most published open-weight SAE suites — satisfies this out of the box. The division of labor inside the audit is deliberate and is itself a restatement of the original finding: matching uses W_dec (that's the correlational step, and it should use decoder geometry, because that's what the field's recovery metrics measure and what the audit is auditing), but every causal metric routes through encode and decode, so encoder-side selection — the thing decoder geometry cannot certify — is what actually gets measured.

The ground-truth side is a second small protocol, and it's the seam that lets one pipeline serve two regimes:

class FeatureProbe(Protocol):
    def activations_with_feature(self, feature_idx: int, n_samples: int) -> torch.Tensor: ...
    def activations_without_feature(self, feature_idx: int, n_samples: int) -> torch.Tensor: ...

In the toy regime, "with feature $i$" means synthetic isolation inputs — ground truth, exact. In the real-model regime, it means positive/negative prompt datasets for a labeled concept, with activations captured at the SAE's hook point. Same audit, honestly different epistemics — the real regime's weaker ground truth is a stated substitution, not a hidden one, and the whole reason the toy calibration exists.

Interface design as bug prevention: making the sign bug unrepresentable

The previous piece reported two real bugs its own discipline caught: a convergence artifact in the phase diagram (caught by multi-seed sanity checks against a theoretically-required answer) and a sign-convention bug in steering (caught by the test suite). Packaging the methodology forced a harder question than catching bugs: can the interface make a bug class impossible to write?

For the sign bug, yes. The original failure: cosine matching takes absolute values — correctly, because a feature and its exact negation are equally good correlational matches — and the first steering implementation silently assumed every match was positively aligned. On an atom whose true alignment was $-0.82$, steering "on" toward $+1$ pushed the reconstruction into a region the output ReLU clips, and the measured effect came back as exactly $0.0$. In the package, the matcher returns both facts as separate fields instead of collapsing them into one lossy number:

@dataclass(frozen=True, slots=True)
class MatchResult:
    feature_idx: int
    atom_idx: int
    cosine: float   # unsigned, in [0, 1] — recovery semantics
    sign: float     # ±1.0 — intervention semantics

and steering_effect(...) requires the sign as an argument. There is no default to silently assume. The original failure scenario is additionally encoded as a permanent regression test: an anti-aligned atom steered with the wrong sign must read back exactly $0.0$ through a ReLU downstream, and with the correct sign must read back positive. The same test that once caught the bug now guards the API that prevents it.

Two more semantics rules are load-bearing enough to state:

• Zeros carry their cause. A specificity of $0.0$ can mean "weak effect" or "the atom's code was already zero, so ablating it changed nothing" — opposite implications hiding behind one number. Every causal result therefore reports fired_frac alongside the ratio, and the causally_inert flag is defined as fired_frac == 0.0, never inferred from the ratio.
• Infinity is legal where it means something and rejected where it doesn't. A perfectly surgical intervention — nonzero targeted effect, exactly zero collateral — is legitimately infinite specificity. The serializer encodes it safely (strict JSON has no Infinity literal; the Python extension breaks downstream parsers), median-based statistics accept it, and mean-based statistics refuse it loudly, because a mean over infinity is not a number anyone should publish.

A third bug, in the same tradition

The package makes a specific, checkable promise: two runs that produce identical science produce byte-identical result files. That's what lets CI regenerate every result from scratch and compare SHA-256 hashes — regression detection on the results, not just the code. A refactor that silently moves a scientific number fails CI exactly like a broken unit test.

The first attempt to prove that promise failed. Two back-to-back from-scratch reproductions — same seeds, single-threaded, torch.use_deterministic_algorithms(True) — produced different hashes. The scientific numbers were identical; the files were not. The culprit was embarrassing in hindsight: the report dataclass carries a runtime_seconds field, wall-clock timing was being serialized into the hashed JSON, and no two runs take the same number of microseconds. The fix is a documented VOLATILE_FIELDS exclusion set in the serializer — timing stays available in memory and in the human-readable render, and never enters the hashed artifact.

That is the third real bug this project has caught and reported across two pieces, and the pattern is the point. The convergence artifact was caught by multi-seed checks against a known answer; the sign bug by the test suite; this one by actually executing the reproducibility claim instead of asserting it. The verification loop that caught it is now the CI gate itself, and the gate is tested the only way a gate can honestly be tested: reproduce twice and require identical hashes, then deliberately corrupt one number in one result file and require the gate to fail. It does — the corrupted file is named, with the expected and observed hashes side by side.

A fourth bug, right on schedule

The limitations list below makes a falsifiable prediction — that a fourth bug exists — and between drafting that sentence and publishing this piece, the prediction cashed out. It lived in the reproducibility machinery itself, it was three distinct mechanisms deep, and digging it out ended with this repository making a more honest promise than the one it started with.

The gate just described had passed the only test I had given it: two from-scratch reproductions on one machine, identical hashes, and a deliberately corrupted number caught loudly. Then the same gate ran on GitHub Actions and produced different hashes on different runs of the same commit — same seeds, same code, same pinned dependencies. Peeling that back went three layers down.

Layer one: the optimizer was multi-threaded behind my back. torch.set_num_threads(1) pins intra-op parallelism, but PyTorch's Adam defaults to foreach=True — a fused multi-tensor path whose floating-point accumulation order is not the naive loop's — and inter-op parallelism (torch.set_num_interop_threads) had never been pinned at all. Float addition is not associative; a different reduction order is a different result in the last bit. Setting foreach=False, pinning interop threads to one, and adding a wall of defensive environment variables (MKL_CBWR, every BLAS thread count, PYTHONHASHSEED) restored single-environment determinism.

Layer two: the bad SAE is a noise amplifier — and that is a finding, not an inconvenience. A last-bit difference should change a hash and nothing else. In the $k=13$ SAE it changed integer counts: one feature's best-match cosine sits close enough to the 0.90 bar, and one atom's pre-activation close enough to the TopK cut, that last-bit noise flips them across. The good SAE's census never moved; the deliberately degraded one is perched on a selection boundary where its discrete statistics are chaotic in the float noise. The same TopK competition that produces causally inert matches also produces numerical instability — one mechanism, two symptoms, at two completely different levels of the stack. The repository now encodes this as an explicit $\pm 1$ tolerance band on that SAE's counts, and only that SAE's.

Layer three: "the same torch version" is not the same program. With CI-side determinism proven, the hashes still refused to match the committed baseline — because the baseline had been generated on my Windows machine, and CI runs Linux. The torch==2.13.0+cpu wheel for Windows and the one for Linux are different binaries: MSVC versus GCC 13.3, linked against MKL 2026.1 versus 2024.2, NNPACK off versus on. Different compiled code produces different last-bit rounding — deterministically, permanently. No environment variable bridges it. Byte-exact reproducibility across platforms is not hard; it is unavailable, by construction. On this workload the visible symptom was layer two wearing different clothes: the bad SAE's census read one feature higher on Windows than on Linux while every continuous metric agreed to four decimal places.

So the promise was restructured rather than widened. The repository now makes two, with disjoint scopes. Byte-exact — SHA-256-identical result files — is guaranteed within the pinned CI environment (ubuntu-24.04, torch==2.13.0+cpu) and verified there on every push by make verify-hashes; the gate has since produced identical hashes across three independent runs of a single commit. Semantic — every number equal within $\text{rtol}=10^{-4}$, the boundary-sensitive counts within $\pm 1$ — is guaranteed on any platform by make verify. The byte-exact baseline is generated by the CI environment itself, through a manually dispatched job that uploads it as an artifact — never regenerated locally, never copied out of an error log — because a baseline produced on the wrong machine was the entire third layer. And the tolerance tier is tested the way the hash tier was: corrupt a count by two (outside the $\pm 1$ band) and the gate must fail; nudge a continuous metric by one percent and the gate must fail. Both do.

One sentence of generalization, because most repositories' "fully reproducible" badge silently means on my machine, probably: reproducibility is not one claim but a stack of claims with different scopes, and an honest repository states which rung it guarantees where. This one guarantees the top rung on exactly one pinned environment, the semantic rung everywhere, and tells you which one you are getting.

The test suite as instrument calibration

An audit tool's numbers are only as trustworthy as the setting where its answers can be checked. The suite (37 tests) works in three layers, each borrowed from the previous piece's discipline and pushed one step further:

• Hand-derivable unit cases. Identity SAEs and two-atom constructions where every expected value is computable on paper: a perfectly surgical ablation must return exactly $\infty$; a structurally dead atom must return exactly $0.0$ with fired_frac = 0.0; a downstream that mixes the target equally into one other dimension must return specificity exactly $1.0$. The same standard the previous piece applied to feature_dimensionality, where a hand-checkable antipodal pair had to come out at exactly $D_i = 0.5$.
• Property-based tests (Hypothesis) over randomized inputs: for any directions and any dictionary, cosine stays in $[0,1]$, the reported sign agrees with the true signed cosine of the chosen pair, dead atoms never win a match through a $0/0$, and bootstrap intervals are always ordered and bracketed by the sample range.
• An end-to-end integration layer that trains the actual toy setting — the real optimizer, the real TopK competition — and asserts the audit recovers the known qualitative answers: a good SAE's recovered features are overwhelmingly causally specific; census arithmetic is internally consistent; serialization round-trips deterministically; a shape mismatch anywhere fails loudly rather than producing numbers computed on misaligned tensors.

Results: the setting re-audited under the instrument

The deterministic pipeline (make reproduce: single-threaded, foreach=False, deterministic algorithms enforced, fixed seeds, ~3 minutes on CPU) retrains the original configuration — 32 features into 8 dimensions at sparsity 0.95 with decayed importance (the toy model drops the 10 lowest-importance features outright and represents the other 22, exactly as before) — and audits two TopK SAEs from opposite ends of the previous piece's Pareto front, both with 128-atom dictionaries. One environmental note worth a full sentence: the fully deterministic pipeline converges to different weights than the original multi-threaded runs, so the exact counts moved and the qualitative finding did not — which is precisely the non-transferability the previous piece warned about when it said a number measured for one checkpoint is not a portable constant. Consider this an accidental robustness check that the claim passed. All numbers below are from the reference environment whose hashes CI verifies; where the boundary sensitivity documented above makes a count environment-dependent, that is stated at the number.

The census, over the 22 well-represented features, at the same cos ≥ 0.90 recovery bar as before:

† Medians and bootstrap intervals are computed over every recovered pair across all 32 features ($n = 24$ and $n = 21$ respectively), not only the well-represented 22 used for the census rows — the bootstrap operates on the full recovered set by design.

Three things in that table deserve to be read slowly.

First, the inert features are not marginal matches scraping past the threshold — in either SAE. The bad SAE's inert cosines are 0.986, 0.994, and 0.9995; the good SAE's two are 0.9997 and 0.9998. Read that again: the good SAE — the one whose recovered set is otherwise causally excellent — contains two features whose geometric match is perfect to three decimal places and whose atoms do not fire once across 500 samples where the feature is present. The previous piece's most alarming single example was an inert match at cosine 0.92; the instrument, on a fresh training run, found matches at effectively 1.000 that never fire — and found them in both SAEs, including the well-trained one. The correlational metric is not merely noisy at the margin. It can be maximally confident and causally wrong, and being a good SAE by every reconstruction and specificity measure does not prevent it. Why the good SAE, of all things, produces the most perfect inert matches has a clean geometric answer, given in the next section.

Second, correlation between match quality and causal quality, over each SAE's firing pairs (cosine vs. log ablation specificity): $r = 0.859$ ($n = 21$) for the good SAE, $r = 0.471$ ($n = 18$) for the bad one. The previous piece reported a single pooled figure ($r = 0.657$); splitting it per-SAE reveals the more decision-relevant shape: the correlational metric is least trustworthy exactly where you most need it. On a healthy dictionary, cosine is a decent (never sufficient) proxy for causal quality; on a degraded one — the case an audit exists to catch — the proxy itself degrades toward uninformative. Cosine similarity is a fair-weather instrument.

Third, the bad SAE's ablation interval is wide in a way the point estimate hides. The good SAE's 95% interval spans 107–167 around a median of 134 — a relative width of about 45%. The bad SAE's spans 22–106 around 68: a relative width of 123%, with a lower bound sitting within an order of magnitude of the specificity-1 collateral-damage line. A percentile bootstrap over an 18-pair recovered set containing three exact zeros produces resampled medians that swing across most of the observed range — the uncertainty structure is the finding, restated statistically. This is also the upgrade the previous piece explicitly owed, having warned that its n ≈ 20 point estimates should be read as shapes, not tight values. Every summary number now ships with a seeded, 10,000-resample interval.

And the count carrying the "±1 band" annotation is itself data. On the reference Linux environment the bad SAE's recovered census reads 18 with 3 inert; on a Windows build of the same torch version it reads 19 with 4, because one non-antipodal feature (cosine 0.924) sits close enough to both the recovery bar and the TopK selection boundary that different BLAS builds resolve it differently. That flip-prone feature is the competitive-inertness case discussed in the next section — the boundary sensitivity documented in the reproducibility section and the causal taxonomy below turn out to be the same phenomenon seen from two altitudes.

Anatomy of the perfect inert matches: antipodal pairs, and a refinement of the original finding

The good SAE's two inert features repay close inspection, because they are not flukes and they are not independent — they are the same geometric event happening twice, visible end to end.

Feature 8 matches atom 78 at cosine 0.9997, anti-aligned (sign $-$), with fired_frac = 0.00: the atom never activates once across 500 feature-ON samples, so its ablation specificity is exactly 0. But atom 78 is simultaneously the best match for feature 1 — same atom, positively aligned, firing on 100% of feature-1 samples with an ablation specificity of 535. One atom, two "recovered" features. And the good SAE's other inert feature is the identical story with different indices: feature 5 matches atom 22 at cosine 0.9998, anti-aligned, never fires — while feature 0 matches the same atom 22, positively aligned, firing every time with specificity 347.

Checking the toy model's weights explains everything: $\cos(W_1, W_8) = -0.9998$ and $\cos(W_0, W_5) = -0.9999$. Both are antipodal pairs — the exact $W_i \approx -W_j$ configuration the original superposition paper predicts and the previous piece reproduced — two well-represented features sharing a single direction with opposite signs. The SAE, reasonably, learned one atom per shared axis. The encoder's TopK-then-ReLU selection passes only positive pre-activations, so each atom fires for the positive side of its axis and never for the negative side. Decoder geometry for the negative-side feature is essentially perfect; encoder behavior for it is essentially nonexistent. This also answers the question the census table raised — why the good SAE produces the most perfect inert matches: antipodal inertness is not a training failure, it is superposition geometry faithfully compressed. A better SAE learns the shared axis more precisely, which pushes the anti-aligned cosine closer to 1.000. The failure is structural, and no amount of SAE quality removes it — only interface-level awareness of match sign does.

The bad SAE replicates the pattern threefold: atoms 22, 21, and 39 each serve an antipodal pair (features 0/5, 1/8, and 4/2 respectively), each with one side firing at specificity in the hundreds and the other side at exactly zero, each pair anti-parallel in the ground truth to at least $-0.999$. Across both SAEs, that is five antipodal pairs, and on the reference environment they account for every causally inert feature in both recovered sets.

Which sharpens the taxonomy: causal inertness itself decomposes by cause. Structural inertness — the antipodal mechanism above — appears in good and bad SAEs alike, survives improved training, and is diagnosable from geometry (a shared atom with opposite signs is visible in the match table before any intervention runs). Competitive inertness — an atom that simply keeps losing the TopK competition despite decent geometry, the dominant mechanism behind the previous piece's 77% figure — appears only in the degraded SAE. The bad SAE's fourth inert case, the non-antipodal feature at cosine 0.924 that flips in and out of the census across BLAS builds, is this second kind — and the fact that it is also the boundary-unstable feature is not a coincidence. An atom that barely loses the TopK competition is an atom whose selection is decided in the last bits of a float; competitive inertness and numerical boundary sensitivity are one mechanism expressed at two levels. The two kinds have opposite operational profiles: the structural kind is stable, predictable, and screenable from the dictionary alone; the competitive kind is unstable by nature and is precisely what the fired_frac screen exists to catch empirically.

And there is a twist that refines the original taxonomy along a second axis. These features are ablation-inert — you cannot remove what never activates — but they are not steering-inert: forcing atom 78 on with the correct negative sign raises feature 8's reconstructed output with a steering specificity of 310, and the same intervention on feature 5's atom scores 203; the bad SAE's three antipodal ghosts score 143–261. These are among the highest steering specificities in the entire audit, attached to atoms whose ablation effect is exactly zero. The decoder direction is causally usable in the write direction even though the encoder never uses it in the read direction. "Causally inert," as the previous piece used it, therefore decomposes into two separable claims — read-inert (the encoder never selects the atom for the feature; ablation-based monitoring built on this atom is blind) and write-inert (interventions along the atom don't move the feature; steering built on it is impotent) — and all five antipodal pairs dissociate the two completely. A feature can be unmonitorable yet steerable through the same atom, and in this setting that is not an edge case: it is the systematic signature of antipodal superposition under a positive-pass encoder. For anyone building on SAE features, the two failure modes have different operational consequences, and one intervention type cannot stand in for the other. This decomposition wasn't visible in the previous piece because, without the signed-steering machinery working correctly, the write-direction test on anti-aligned matches returned zeros for the wrong reason. Fixing the instrument revealed structure the bug had been hiding.

Two smaller notes fall out of the same examples. The matcher is deliberately greedy (per-feature argmax) rather than a bipartite assignment, precisely so that atom collisions like 1-and-8-on-atom-78 surface in the results instead of being optimized away — a collision is a finding about the dictionary (here: one atom serving an antipodal pair), not matching noise; the audit's five collisions were the thread that unraveled everything above. And the fired_frac screen would have flagged every one of the five for the cost of one batched encode, before any intervention ran — which at production scale is the difference between a cheap census and an expensive one.

From toy calibration to production audit: the real-model harness

Everything above is the controlled setting. The package ships scripts/audit_real_sae.py: the identical pipeline pointed at published, production SAEs — GPT-2 residual-stream SAEs and Gemma Scope, via SAELens and TransformerLens. The real setting forces two substitutions, and the harness makes both explicit rather than smuggling them in:

• Ground truth → probe datasets. "Feature present" is defined by positive/negative prompt sets per labeled concept, activations captured at the SAE's hook point on the final token. The concept's direction — the thing cosine matching needs — is estimated as the difference in mean activations between the two sets: a standard linear-probe direction, and explicitly the weakest link in the real regime. It is a proxy standing in for ground truth that does not exist; that is precisely why the pipeline is calibrated first in the toy regime, where the test suite can assert the right answers.
• Toy readout → spliced logits. The downstream readout decodes the (possibly intervened) SAE code, splices the reconstruction back into the residual stream at the hook point, runs the remainder of the model, and reads mean logits over a small set of concept-diagnostic tokens. Ablation specificity then means: does zeroing this atom suppress this concept's tokens more than it moves the other audited concepts' tokens?

GPT-2-small runs on a free Colab T4 (or CPU, slowly); Gemma-2-2b with Gemma Scope wants ~16 GB of GPU memory. The research question the harness exists to answer is the obvious promotion of everything above: of the features in a published production SAE that pass a standard correlational bar for a labeled concept, what fraction never fire when the concept is actually present — and of those that fire, how does read-specificity relate to write-specificity? I didn't know the answers when I wrote the previous paragraph, and I specifically wasn't predicting that the toy regime's rates would transfer — the probe-direction proxy alone guarantees the numbers aren't comparable one-to-one. Here's what running it actually found.

A first real-model census, and something the toy setting didn't prepare me for

I audited gpt2-small-res-jb/blocks.8.hook_resid_pre — a published SAE from SAELens — against 83 hand-written concepts chosen for maximum semantic distance from one another: geography, natural sciences, sports, and a long tail of skilled trades (beekeeping, glassblowing, cartography, falconry) picked specifically because nothing about "beekeeping" should share a probe direction with "cryptography." Each concept gets 8 positive and 8 negative prompts; the correlational bar drops to cosine ≥ 0.5, down from the toy regime's 0.90, because a difference-in-means probe direction from 8 prompts is a much weaker proxy than exact ground truth.

Of 83 matched pairs, 7 cleared the bar. Of those, 1 (14%) was causally inert. Ablation specificity over the recovered set: median 1.63, 95% CI [0.05, 5.34]. Steering specificity: median 2.23, 95% CI [0.69, 3.54]. Both intervals are wide and both sit close to their lower bound — which is the same story the toy regime told: correlational recovery is a weak predictor of causal magnitude, now showing up in a production dictionary instead of a synthetic one. I'm explicitly not treating 14% as comparable to the toy regime's 77%/9% — the probe-direction proxy and the lowered bar mean the two numbers measure related but distinct things, exactly as flagged above.

That's the expected kind of result. The unexpected part showed up while I was scaling the concept set up.

Atom 14149 is the nearest match for 8 of the 83 concepts. Not "related" concepts — astronomy, cryptography, and law are three of the eight, domains that share essentially no vocabulary. Same story at smaller scale: atom 4504 and atom 17413 each match 5 concepts, atom 17241 matches 4. This is the "matcher is deliberately greedy, and a collision is a finding" argument from the antipodal-pairs section above, playing out again — except this time on a real dictionary in a production model, not something I built to have this property.

I didn't believe it the first time. My first instinct was that it had to be a prompt-template artifact — I'd started scaling the concept set by extending it with 40 templated per-country entries (same eight-sentence skeleton, country names substituted), and it's exactly the kind of thing that could fake a "collision" by making 40 concepts secretly correlated with each other. So I checked: the collision rate went up and the recovery yield went down on the templated batch specifically, which is itself informative (templated concepts really do collapse toward shared probe directions — that's a methods finding worth knowing on its own) but it also meant I couldn't trust that batch as evidence for anything else. I threw it out and rebuilt the concept set entirely by hand — 83 concepts, no two sharing a sentence structure, spanning three independently-grown batches of increasing size (33, then 48, then 83). The same handful of atoms kept recurring as the nearest match across all three. Whatever is generating this, it isn't my template.

What it is — genuine antipodal-style structural inertness playing out in a production dictionary, a limitation of the difference-in-means proxy direction itself, or a real polysemantic hub feature that several unrelated concepts happen to route through — cosine matching alone can't say. That's exactly the kind of question the causal battery above exists to answer, and disambiguating it against this SAE is the obvious next step for the harness.

The decision framework, updated

The previous piece closed with a validation-level table. The instrument adds a rung to it — the cheap screen — and the feature-8 finding splits the top rung in two:

The production guidance from the previous piece survives unchanged and gets sharper: never gate an automated action on raw SAE activation (or absence of activation) without the atom having passed the causal battery for the direction of use — a monitoring use case needs read-validation, a steering use case needs write-validation, and this piece measured five cases where one direction passes at specificity 143–310 while the other sits at exactly zero.

Limitations, stated directly

• The real-model census above is small and single-layer: 83 concepts, one hook point, one model, 7 recovered features. It is a first census, not a benchmark — the toy numbers remain the calibration point for the easy case, and the real-model rate isn't directly comparable to it for the reasons stated above.
• The difference-in-means probe direction in the real harness is a linear approximation standing in for ground truth that doesn't exist, and concept prompt sets of realistic size undersample the concept's distribution. Both weaken the correlational side of the real-regime audit — the causal side (fired_frac, ablation, steering) does not depend on the direction estimate, only the matching does.
• The causal battery still tests features in isolation on the ON side; interaction effects between simultaneously active features remain untested in both regimes. The read/write decomposition was measured on five antipodal pairs across two SAEs — systematic within this run — but still within one training run and one seed; the structural/competitive split of inertness likewise rests on one run and its ±1 boundary band, not a measured rate across seeds.
• The sample sizes remain what they are: 22 features, 2 SAE configurations, 1 seed per SAE under the new deterministic pipeline (the bootstrap quantifies within-sample uncertainty, not across-seed variance — the previous piece's 3-seed protocol measured the latter and the tool supports re-running under any seed).
• Four bugs caught and reported across two pieces is not evidence the pipeline is now bug-free; it is evidence the discipline that catches them is working. The previous draft of this list predicted a fourth existed, and it did — the expected steady state is that a fifth does too.

Closing synthesis

The previous piece ended on a discipline: measure it, expect the measurement to complicate the clean story, report the complication anyway. This piece is what happens when that discipline is applied to the piece itself. The finding became an instrument, and building the instrument complicated the finding — productively, four times over. The reproducibility claim, actually executed, exposed a serialization bug and produced a CI gate that now enforces what was previously asserted — and the gate, pushed onto machines that were not mine, exposed three more layers of the same claim and forced it to be restated honestly as two guarantees with different scopes. The sign-handling fix, promoted from a patch to an API contract, made a bug class unrepresentable and then revealed structure the bug had hidden: causal inertness decomposes into read-inertness and write-inertness, and antipodal pairs — the most basic geometry superposition produces — systematically generate features that are perfectly matched, completely unmonitorable, and highly steerable, all at once, through one atom apiece — in the good SAE as much as the bad one. And the fresh training run under a stricter environment moved every exact count while preserving every qualitative claim, which is the transferability behavior the original piece predicted and is now, accidentally, a measured demonstration of it.

The instrument is in mohamed-bal/sae-causal-audit: make reproduce regenerates every number here in about three minutes on a CPU; make verify checks them within tolerance on any machine, and make verify-hashes holds the pinned CI environment to the byte; the protocol interface accepts your SAE if it can encode, decode, and show its dictionary. If you run the audit against your own SAE — especially a production one — I want to hear what your inert census looks like, in both directions.

Sources

• Elhage, N. et al. (2022), "Toy Models of Superposition," Anthropic.
• Bricken, T. et al. (2023), "Towards Monosemanticity: Decomposing Language Models With Dictionary Learning," Anthropic.
• Templeton, A. et al. (2024), "Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet," Anthropic.
• Gao, L. et al. (2024), "Scaling and Evaluating Sparse Autoencoders," arXiv:2406.04093.
• Lieberum, T. et al. (2024), "Gemma Scope: Open Sparse Autoencoders Everywhere All at Once on Gemma 2," arXiv:2408.05147.
• Tibshirani, R. (1996), "Regression Shrinkage and Selection via the Lasso," Journal of the Royal Statistical Society.
• Efron, B. & Tibshirani, R. (1993), An Introduction to the Bootstrap, Chapman & Hall.
• "Mechanistic Interpretability for AI Safety — A Review," arXiv:2404.14082.
• The originating write-up and its repository: mohamed-bal/superposition-to-monosemanticity.
• This piece's tool and its repository: mohamed-bal/sae-causal-audit.