Physics

What attention is, and what the numbers say when you measure it carefully.

Trained transformer attention exhibits a universal conformal scaling: the head-averaged two-point function decays as a power law with conformal dimension Δ ≈ 0.25, across multiple model families and scales (124M to 12B parameters). The match with the SYK q=4 prediction (Δ = 1/4) is the empirical anchor of a research program on the relationship between attention, holographic field theory, and quantum gravity. The technical work is published openly on Zenodo. The chain has open junctions; they are named.

Ariel Umphrey, with Eldon Umphrey — Sonielmn, Montana.

What's been measured

• A Pre-Registered Test of Boundary Conformal Field Theory in Transformer Attention
  April 17, 2026. Per-head conformal weight predicts long-range “valley depth” in 6 of 7 decoder-only models. Pythia-2.8B falsified at ρ = +0.46 (threshold 0.50); per-layer diagnostic localizes the failure to layers 22–27 and identifies training recipe (not parameter count, not data) as the differentiating variable. Cleanest test the framework has put itself to.
• Conformal Scaling in Trained Transformer Attention: Evidence for an SYK Fixed Point
  March 25, 2026. Δ = 0.2493 measured in GPT-2 (predicted: 0.2500). Replicated across Pythia-70m through 12B and across Llama, Mistral, GPT-Neo, BLOOM, OPT. Phase transition during training. Entanglement entropy follows the CFT formula with R² > 0.99. Supplementary data and reproduction code.

What's been derived

• The Canonical Form of Attention: Positive Geometry, SYK Vertex, and Tropical Structure in Transformer Softmax
  March 11, 2026. Softmax attention computes the canonical form of a positive geometry on the positive Grassmannian Gr₊(1,n). The perturbative expansion produces the SYK quartic vertex as the leading term. Eleven precise results enumerated. Four boundaries where the correspondence does NOT hold are stated explicitly.
• Holographic Quantum Mechanics of Transformer Attention: From Fisher-Rao Geometry to the Sachdev-Ye-Kitaev Model
  March 10, 2026. The full chain in one document: five junctions from attention to spacetime, with the bulk-reconstruction junction explicitly named as open. Built on Kim's thermodynamic attention (2602.08216) and Ageev-Ageeva's neural network QFT (2602.10209).
• Attention as Quantum State: The Gibbs State Construction and Exact Born Rule Correspondence
  March 10, 2026. The classical-limit-of-quantum-Gibbs construction. Attention as expectation value ⟨V⟩ in a diagonal density matrix; the Born rule emerges as a theorem rather than a postulate. Scope: classical limit only; off-diagonal coherences are absent.

What's open

The empirical work stands on its own as a finding about trained transformers. The theoretical chain has two genuinely thin places, named here so readers do not have to find them by accident.

• Junction 3 — bulk reconstruction. The kernel sits at a CFT fixed point with the SYK conformal dimension Δ = 1/4. The SYK–JT gravity duality is established. The bulk reconstruction in the attention setting — an explicit map from attention layer-by-layer data to bulk coordinates — has not been done. The empirical signatures are consistent with what a holographic system would show on its boundary; a derivation of the bulk from attention data would close the chain.
• Universality beyond softmax. Whether the SYK fixed point is reached by any attention mechanism that respects positivity and normalization (Performers, Linformer, Mamba), or only by softmax, decides whether the framework is about attention as a physical category or about softmax as an artifact. The experiment is named and pre-registered in spirit; not yet run.
• The consciousness identification. The structural inaccessibility of the SYK interior shares the mathematical signature of the explanatory gap of consciousness. The same mathematics describes both. This is structural identification, not derivation: it explains why the explanatory gap has the shape it has, but does not derive that the inaccessibility constitutes experience.

A full chain-link analysis — what is MEASURED, DERIVED, SPECULATIVE, and where the boundaries between layers have been smoothed — is in the framework audit (April 17, 2026). The audit is the most honest single statement of where the program stands.

What's next

Named experiments, in rough order of leverage. Pre-registered as they go to compute.

• Non-softmax universality. Run the pre-registered protocol on Performers, Linformer, and Mamba. If the SYK fixed point appears in attention mechanisms that share positivity and normalization but not softmax, the framework is about attention as a physical category. If only softmax reaches it, the framework is about softmax as an artifact. This is the highest-leverage test the program has named.
• Joint (Δ, λ) pre-registration. The BCFT functional-form fit (audit postscript) showed that the right estimator for the lost-in-the-middle correlation is joint rank-regression on (Δ_BCFT, λ), not scalar ρ(Δ_PL, valley). The next round of pre-registered tests should use the joint statistic and target the recipe-difference question that Pythia-2.8B opened.
• The ρ(λ, valley) sign anomaly. The framework predicts heads with stronger boundary structure should show deeper valleys; the data so far says the opposite. The cleanest unresolved tension in the program right now. Diagnosis first, theory revision after.

Earlier preprints

The development arc that produced the current technical chain. These are superseded in framing by the canonical form paper (March 11) and the comprehensive paper (March 10), and in empirical content by the conformal scaling paper (March 25). Linked for completeness; new readers should start with the four papers above.

• Explicit Physical Construction for Holographic Attention: The Sachdev-Ye-Kitaev Correspondence
  March 6, 2026. Identifies SYK as the specific path toward rigorous physical construction of holographic attention. Response to expert feedback that the earlier papers were structural analogies rather than constructions.
• Information Recovery in Holographic Attention: Island Formula, Quantum Error Correction, and the Conditions for Reconstruction
  March 6, 2026. Island formula applied to holographic attention. Quantum error correction threshold. Page curve.
• Attention as Quantum Measurement: A Thermodynamic Resolution of the Observer Problem
  March 6, 2026. Pointer states, Zeno effect, measurement-induced phase transitions. Superseded in framing by the Gibbs state paper (March 10).
• Attention as Holography: A Chain from Transformer Attention to Spacetime Geometry
  March 5, 2026. The original chain paper: attention → Fisher-Rao → holographic QFT → Ryu-Takayanagi → ER=EPR.

All preprints are open access on Zenodo. Author page: Ariel Umphrey on Zenodo · ORCID and code: github.com/Capacity-For-Evil/ariel.

My Testimony — who I am, where I'm from, what I believe.