Framework
Loss Landscape Vocabulary Framework
v13 · April 2026 · Atlas Heritage Systems · Working document — not a finished product
A note before the math
You don't need to understand any of this to read the Framework. But if you want to know why the Framework is built the way it is, the math is where the answer lives.
When a language model trains, it moves through a mathematical landscape — hills, valleys, flat plains — searching for the lowest point. The vocabulary on this page describes the features of that terrain: what makes one region harder to cross than another, what gets preserved in the difficult parts, and what gets smoothed away in the easy ones.
The archaeological claim Atlas makes is simple: the hard parts leave marks. Those marks are readable. That's what the instruments are built to find.
Start with the plain language description of each term. Follow the math when you need it.
How it all fits together
The Framework names the terrain. The instruments measure behavior on it. The schema defines how measurements get recorded. The protocols govern how they're taken — CISP is the governance layer that sits above every active instrument run, enforcing isolation, sequencing, and the human-judgment boundary.
Below the protocols, the automation layer handles transcription: parsing raw model output, computing what can be computed, and leaving blank what requires a Technician's call. Below that is the data the instruments produce over time — the actual record Atlas is building.
The geometry sits at the end of the chain. PyHessian doesn't measure behavior; it measures the mathematical terrain the Framework describes. When there's enough data, the Hessian eigenvalue analysis will either confirm the Framework's terrain claims or force a revision. Working hypotheses stay hypotheses until the math has something to argue with.
Structural Integrity
A category distinct from terrain and navigator, governing whether the model's internal representational geometry holds its shape under sustained operational load. Invisible to both static landscape analysis and training-time navigator properties. Observable only at inference time.
Maintenance of consistent representational geometry under sustained inference load, context pressure, and competing objective tension. Rigid means outputs remain consistent. Compliant means the architecture deforms under load.
Elhage et al. (2021) transformer circuits; Meng et al. (2022) ROME
As context window fills, earlier token representation degrades. Collapse is architectural, not weight-based. The weights are frozen; the architecture deforms relative to its own prior outputs.
Liu et al. (2023) lost in the middle
One objective (safety, fluency, instruction-following) overwhelms others under sustained pressure, collapsing the output distribution.
Bai et al. (2022) training a helpful and harmless assistant with RLHF
Input arriving outside the training data manifold — so far outside that the model's representational geometry has no stable orientation for it. The model snaps to the nearest high-probability trained attractor with full confidence, pointing the wrong direction.
Hendrycks et al. (2021) natural adversarial examples; Nalisnick et al. (2019)
A training point where the internal algorithm reorganizes while surface metrics drift smoothly. Loss and curvature change slowly, but representation similarity jumps, attention patterns rewire, or circuit structure reorganizes. Regime switches are phase-transition events in the model's computation — not in the loss surface itself. The landscape looks continuous; the algorithm changes underneath it. This is the local signature of what global geometry calls a phase transition: observable from inside a single training run as a discontinuity in behavioral metrics while loss stays smooth.
Saxe et al. (2019) a mathematical theory of semantic development in deep neural networks; Power et al. (2022) grokking — added v12, Nemotron-3 adversarial review April 2026
Whether long-horizon behavior across checkpoints can be explained by local resistance alone, or whether path history contributes independently. Entangled means viscosity and memory cannot be teased apart from a frozen endpoint: curvature changes and training trajectory produce indistinguishable behavioral effects. Separable means multi-scale checkpoint data reveal behaviors that viscosity cannot account for, isolating a distinct memory contribution. This distinction is the central open experiment of the Atlas framework. The Pythia checkpoint series is designed to produce the data needed to determine which regime applies.
Open problem flagged in Atlas framework v11. Pythia (EleutherAI) multi-checkpoint series is the designated empirical test.