Training-Free & Test-Time Optimization / Training — SOTA Cheat Sheet

Methods that improve a frozen (or barely-touched) model at inference — no training run required. Decoding, activation steering, weight merging, speculative decoding, guidance; test-time compute & search; test-time adaptation (TTA); test-time training (TTT).

With derivations — every technique carries its governing equation, and the load-bearing results are proved.

Updated August 2026 with 2025–2026 SOTA — new entries marked ★. Method names link to their primary papers (arXiv / official page).

August 2026 · Updated Edition

Contents

1. Taxonomy: The Test-Time Landscape
2. Training-Free Inference — Decoding-Time Methods
3. Training-Free Control — Activation Steering & Representation Engineering
4. Training-Free Weight Arithmetic — Model Merging
5. Training-Free Acceleration — Speculative Decoding
6. Training-Free Generation Control — Diffusion Guidance
7. Test-Time Compute — Sampling, Search & Verifiers
8. Test-Time Compute — Reasoning-Model Scaling, RL & Theory
9. Test-Time Adaptation (TTA)
10. Test-Time Training (TTT)
11. TTT as Sequence Modeling — TTT Layers & Test-Time Memory
12. Test-Time Optimization Without Weight Updates
13. Unifying View & Practical Guidance

Appendix A: Twenty-Five Things to Memorize Appendix B: Decision Guide — "Which Test-Time Method?" Appendix C: Formula Sheet Appendix D: Year-by-Year Milestones

1. Taxonomy: The Test-Time Landscape

All of these techniques share one premise: the expensive training run is over; we now spend effort at inference. They differ along three axes.

Axis 1 — what is optimized (nothing → everything):

• Nothing (pure inference cleverness): decoding rules, guidance, speculative decoding, ensembling.
• Inputs / prompts: test-time prompt tuning, textual-gradient optimization.
• Activations: steering vectors, representation engineering.
• A separate state/model: TTT-layer hidden state, cache adapters (TDA).
• A few parameters: BN affine (TENT), LoRA per task (ARC-TTT), prompt vectors (TPT).
• All parameters: MEMO, online TTT.
• Weights, arithmetically (no gradients): model merging.

Axis 2 — what signal drives the optimization:

• None (fixed rule) · self-consistency / agreement · prediction entropy · self-supervision (reconstruction, rotation, contrastive) · a verifier / reward · preference feedback.

Axis 3 — how the compute is spent: parallel (sample many, aggregate) vs sequential (revise/iterate/adapt over steps). This axis governs test-time-compute scaling (§7–8).

Key

Two broad families. Training-free inference (§2–§6) changes how you run a fixed model — no optimization loop, or an optimization that touches only inputs/activations. Test-time optimization/training (§7–§12) runs an actual optimization at inference: search over samples (test-time compute), a few gradient steps on unlabeled data (TTA/TTT), or a textual/latent optimization loop. The recurring constraint everywhere is the signal: a method can only push the model as far as its driving signal (agreement, entropy, self-supervision, verifier) is correct.

2. Training-Free Inference — Decoding-Time Methods

These reshape the token distribution at decode time; the network is frozen. Common shape: a contrast between two distributions, or a risk objective over samples.

2.1 Contrastive Decoding

Maximize the log-likelihood gap between a strong "expert" and a weak "amateur" LM, restricted to tokens the expert finds plausible:

\[\text{score}(x_i) = \log p_{\text{EXP}}(x_i \mid x_{<i}) - \log p_{\text{AMA}}(x_i \mid x_{<i}), \quad x_i \in \mathcal{V}_{\text{head}},\]

\[\mathcal{V}_{\text{head}} = \Big\{ x : p_{\text{EXP}}(x \mid x_{<i}) \ge \alpha \max_w p_{\text{EXP}}(w \mid x_{<i}) \Big\} \ (\alpha\approx0.1).\]

The amateur cancels generic failure modes (repetition, genericness) that both models share, leaving the expert's distinctive competence. The adaptive-plausibility mask \(\mathcal{V}_{\text{head}}\) prevents the contrast from amplifying implausible tokens.

2.2 DoLa

Contrast the final layer against an early ("premature") layer of the same model — factual knowledge tends to mature in later layers:

\[\log p(x) = \log\text{softmax}\big(q_N(x) - q_M(x)\big), \quad M = \arg\max_{j\in\mathcal{J}} \mathrm{JSD}\big(q_N \,\|\, q_j\big),\]

with \(q_\ell\) the logits obtained by early-exiting at layer \(\ell\) and \(M\) the premature layer chosen dynamically by maximal Jensen–Shannon divergence. Reduces hallucination with no training and no second model.

2.3 Contrastive Search

Decode by trading model confidence against a degeneration penalty (self-similarity to prior hidden states), over top-\(k\) candidates:

\[x_t = \arg\max_{v\in \mathcal{V}^{(k)}} \Big\{ (1-\alpha)\, p_\theta(v\mid x_{<t}) - \alpha \max_{1\le j<t} \cos\big(h_v, h_{x_j}\big) \Big\}.\]

The penalty keeps the representation space isotropic, avoiding repetitive loops without sampling.

2.4 Context-Aware Decoding (CAD)

A pointwise-mutual-information contrast that makes the model trust its context (retrieved evidence) over its priors:

\[y_t \sim \text{softmax}\Big[(1+\alpha)\,\text{logit}_\theta(y_t\mid c, x, y_{<t}) - \alpha\,\text{logit}_\theta(y_t\mid x, y_{<t})\Big],\]

with context \(c\) and query \(x\); \(\alpha=0\) recovers standard decoding. Cuts context-ignoring hallucinations in RAG/summarization.

2.5 Classifier-Free Guidance for LMs

Import diffusion-style CFG (§6) into autoregressive LMs to sharpen prompt adherence:

\[\log \tilde{p}(w\mid \text{prompt}) \propto \log p(w\mid \varnothing) + \gamma\big[\log p(w\mid \text{prompt}) - \log p(w\mid \varnothing)\big], \quad \gamma>1.\]

Structurally identical to CAD (context = prompt): both extrapolate along the log-prob difference induced by conditioning.

2.6 Minimum Bayes Risk (MBR) decoding

Decision-theoretic decoding: instead of the most probable hypothesis (MAP), pick the one with highest expected utility against a sampled pseudo-reference set \(H\):

\[\hat{y} = \arg\max_{h\in H} \sum_{y\in H} p(y\mid x)\,u(h,y) \approx \arg\max_{h\in H} \frac{1}{|H|}\sum_{y\in H} u(h,y),\]

equivalently minimizing Bayes risk \(R(h)=\mathbb{E}_{y}[1-u(h,y)]\) with utility \(u\) (BLEU, COMET, exact-match). MBR beats MAP when the mode is degenerate — it targets the consensus, not the peak. Frustratingly Simple Decoding is a cheap cousin contrasting against an on-the-fly \(n\)-gram anchor to kill repetition.

Key

Almost every decoding-time method is one of two objects: a contrast \(\log p_A - \lambda\log p_B\) (CD, DoLa, CAD, CFG-LM — amplify what conditioning/depth/scale adds) or a risk/agreement objective over samples (MBR, self-consistency §7.1). No parameters move; only the score does.

3. Training-Free Control — Activation Steering & Representation Engineering

Control behavior by editing the residual stream. Universal form at layer \(\ell\): \(h_\ell \leftarrow h_\ell + \alpha\, v\), where \(v\) is a direction extracted from contrastive data — no weight updates.

• ActAdd / Activation Addition — steering vector from a single prompt pair: \(v = h_\ell(\text{prompt}_+) - h_\ell(\text{prompt}_-)\); add at generation. No dataset, no training.
• Representation Engineering (RepE) — read a concept direction as the top principal component of stimulus-evoked differences \(\{h_\ell(s_i^+)-h_\ell(s_i^-)\}\) (Linear Artificial Tomography), then control by \(h_\ell \leftarrow h_\ell \pm \alpha v\). A general read/write framework for concepts (honesty, harmlessness, emotion).
• Contrastive Activation Addition (CAA) — dataset-averaged steering vector \(v_\ell = \tfrac{1}{|D|}\sum_{(p,n)\in D}[h_\ell(p)-h_\ell(n)]\) over paired completions; more stable than a single pair.
• Inference-Time Intervention (ITI) — shift activations along a probe-identified "truth" direction in the most informative attention heads: \(x_{\text{head}} \leftarrow x_{\text{head}} + \alpha\,\sigma_{\text{head}}\,\theta_{\text{head}}\), heads ranked by probe accuracy. Improves truthfulness at inference.
• Function Vectors and In-Context Vectors — compress an in-context task into a single vector (causal-mediation-selected head outputs, or PCA of target−source hidden states) that triggers the task in a new context without demonstrations — ICL distilled into one additive direction.

Key

Steering treats a concept as a (near-)linear direction in activation space — the linear representation hypothesis. It is the training-free dual of fine-tuning: fine-tuning moves weights to move activations; steering moves activations directly. Cost is one forward pass; the risk is off-target effects when the direction is not truly linear/monosemantic.

4. Training-Free Weight Arithmetic — Model Merging

Combine capabilities by arithmetic on weights — no retraining, no data (or a tiny calibration set). Central object: the task vector \(\tau = \theta_{\text{ft}} - \theta_{\text{pre}}\) (a finetune's weight delta).

• Task Arithmetic — add/negate task vectors: multi-task \(\theta = \theta_0 + \sum_i \lambda_i \tau_i\); unlearn a skill by \(\theta = \theta_0 - \lambda\tau\); even analogies \(\tau_{\text{new}}\approx\tau_a-\tau_b+\tau_c\). Works because finetuning stays in a near-linear (tangent-space / NTK) regime where task vectors approximately superpose.
• TIES-Merging — resolve interference before adding: (1) trim each \(\tau_i\) to its top-\(k\%\) magnitudes; (2) elect a sign per parameter, \(\gamma^p=\text{sign}(\sum_i \hat\tau_i^p)\); (3) disjoint-mean only the entries agreeing with the elected sign: \(\tau_m^p = \tfrac{1}{|A^p|}\sum_{i\in A^p}\hat\tau_i^p\), then \(\theta=\theta_0+\lambda\tau_m\).
• DARE — a preprocessor: randomly drop delta entries with mask \(m\sim\text{Bernoulli}(1-p)\) and rescale \(\tilde\tau=(m\odot\tau)/(1-p)\) so \(\mathbb{E}[\tilde\tau]=\tau\); enables merging many models with little interference.
• Model Soups — average whole finetunes of the same base; greedy soup adds an ingredient only if held-out accuracy does not drop. Uniform soup \(\theta=\tfrac1N\sum_i\theta_i\).
• Fisher-weighted averaging (\(\theta_j^\star=\frac{\sum_i F_{i,j}\theta_{i,j}}{\sum_i F_{i,j}}\), \(F\) = diagonal Fisher) and RegMean (closed-form per-layer merge \(W^\star=(\sum_i X_i^\top X_i)^{-1}(\sum_i X_i^\top X_i W_i)\)) weight parameters by importance / input statistics. SLERP interpolates on the hypersphere, \(\text{slerp}(\theta_0,\theta_1;t)=\tfrac{\sin((1-t)\Omega)}{\sin\Omega}\theta_0+\tfrac{\sin(t\Omega)}{\sin\Omega}\theta_1\) (geometric method, no arXiv origin).

Key

Model merging is "test-time optimization" in weight space with zero gradient steps — you compose behaviors algebraically. It only works because independent finetunes of a shared base live in a roughly flat, near-linear basin; the whole TIES/DARE line is machinery to suppress the interference that breaks that assumption.

5. Training-Free Acceleration — Speculative Decoding

Make a fixed model decode faster without changing its output distribution. A cheap draft \(q\) proposes tokens; the target \(p\) verifies them in parallel with a rejection rule that is provably exact.

5.1 Speculative decoding / sampling

Draft \(x\sim q\); accept with probability \(\min(1, p(x)/q(x))\); on rejection, resample from the normalized residual

\[p'(x) = \frac{\max\big(0,\, p(x)-q(x)\big)}{\sum_w \max\big(0,\, p(w)-q(w)\big)}.\]

Correctness (the token is distributed exactly as \(p\)). Let \(X\) be the emitted token. Using \(\min(a,b)\) for the accept mass and \(\beta=\sum_w(p(w)-q(w))_+\) for the total rejection mass,

\[\Pr[X=x] = \underbrace{q(x)\min\!\big(1,\tfrac{p(x)}{q(x)}\big)}_{\text{accept}} + \underbrace{\beta\cdot p'(x)}_{\text{reject, resample}} = \min\big(p(x),q(x)\big) + \big(p(x)-q(x)\big)_+ = p(x),\]

because \(\min(p,q)+(p-q)_+ = p\) pointwise, and \(\sum_w(q-p)_+=\sum_w(p-q)_+=\beta\) since \(\sum p=\sum q=1\). \(\blacksquare\) So speculative decoding is lossless: same distribution, fewer serial target calls.

5.2 Expected speedup

With per-token acceptance rate \(\alpha\) and \(\gamma\) drafted tokens per block, the expected number of tokens produced per target pass is

\[\mathbb{E}[\text{tokens}] = \frac{1-\alpha^{\gamma+1}}{1-\alpha},\]

so the serial-step reduction grows with \(\alpha\) (how well the draft mimics the target). Realized wall-clock speedup multiplies this by the draft/target cost ratio.

5.3 Draft-model variants

• Medusa — add lightweight extra heads that predict several future tokens; verify candidate trees with tree attention and a typical-acceptance threshold (keep \(x\) if \(p(x)>\min(\epsilon, \delta e^{-H(p)})\)) instead of full rejection sampling.
• EAGLE — autoregress at the feature (penultimate-layer) level plus the shifted token, then verify — higher acceptance, still lossless.
• Self-Speculative Decoding — the model drafts by skipping some of its own layers, then verifies with the full pass; no auxiliary model.
• Lookahead Decoding — draft-model-free: solve the Jacobi fixed point of the autoregressive map in parallel and verify collected \(n\)-grams — exact, no draft net.

6. Training-Free Generation Control — Diffusion Guidance

Steer a frozen diffusion model at sampling time. Base guidance form (noise-prediction \(\varepsilon\)):

\[\tilde{\varepsilon} = \varepsilon_{\text{uncond}} + s\,(\varepsilon_{\text{cond}} - \varepsilon_{\text{uncond}}).\]

(Convention note: CFG is often written \(\tilde{\varepsilon}=(1+w)\varepsilon_c - w\varepsilon_\varnothing\); this equals the above with \(s=1+w\), i.e. their \(w=s-1\). Watch the off-by-one when porting scales.)

6.1 Classifier guidance — the derivation

To sample the posterior \(p(x\mid y)\propto p(x)\,p(y\mid x)\), its score decomposes:

\[\nabla_{x_t}\log p(x_t\mid y) = \nabla_{x_t}\log p(x_t) + \nabla_{x_t}\log p(y\mid x_t).\]

Using the score↔︎noise identity \(\nabla_{x_t}\log p(x_t) = -\varepsilon_\theta(x_t)/\sigma_t\), the guided predictor is

\[\tilde{\varepsilon}_\theta(x_t,t) = \varepsilon_\theta(x_t,t) - s\,\sigma_t\,\nabla_{x_t}\log p_\phi(y\mid x_t),\]

with a separately trained noised-image classifier \(p_\phi\). Scale \(s>1\) samples the sharpened \(p(x)p(y\mid x)^s\).

6.2 Classifier-Free Guidance — implicit classifier

By Bayes, \(\nabla_{x_t}\log p(y\mid x_t) = \nabla_{x_t}\log p(x_t\mid y) - \nabla_{x_t}\log p(x_t)\), i.e. the classifier gradient equals the difference of conditional and unconditional scores. Substituting removes the external classifier:

\[\tilde{\varepsilon} = \varepsilon_\theta(x_t,\varnothing) + s\,\big(\varepsilon_\theta(x_t,c) - \varepsilon_\theta(x_t,\varnothing)\big).\]

One jointly-trained model, dropping the condition at random during training, gives both terms. This is the workhorse of modern text-to-image.

6.3 Label-free, training-free guidance

• Self-Attention Guidance (SAG) — \(\tilde{\varepsilon}=\varepsilon_\theta(x_t)+s(\varepsilon_\theta(x_t)-\varepsilon_\theta(\hat{x}_t))\) where \(\hat{x}_t\) blurs the self-attention-salient regions; guides away from a degraded self.
• Perturbed-Attention Guidance (PAG) — same shape but the "bad" path replaces self-attention maps with the identity \(A\leftarrow I\); improves quality with no condition and no labels.
• Universal Guidance — plug any off-the-shelf loss \(\ell(c,\hat{x}_0(z_t))\) (CLIP, segmentation, face-ID) through the Tweedie clean-image estimate \(\hat{x}_0\): \(\hat{\varepsilon}=\varepsilon_\theta(z_t)+s\nabla_{z_t}\ell(c,\hat{x}_0(z_t))\), with per-step self-recurrence. FreeU reweights U-Net backbone vs skip features in the Fourier domain — a free-lunch quality boost, no params.

Key

Guidance is test-time steering of a generative score: pick a direction (a classifier gradient, a conditional−unconditional difference, a perturbed-self difference, or any differentiable loss through \(\hat{x}_0\)) and extrapolate along it. It trades diversity for fidelity/control and costs one extra network eval per step — the generative analogue of activation steering (§3).

7. Test-Time Compute — Sampling, Search & Verifiers

Now we spend inference compute on an actual search/optimization over outputs. The model is frozen; the procedure around it does the work.

7.1 Sampling-based scaling

Self-Consistency marginalizes the latent reasoning path by sampling \(N\) chains and majority-voting:

\[\hat{a} = \arg\max_a \sum_{i=1}^{N}\mathbb{1}[a_i=a], \qquad (r_i,a_i)\sim p(\text{reason, answer}\mid q).\]

Why voting works — error bound. For a (binary) decision correct per-sample with probability \(p>\tfrac12\), the majority is wrong only if \(\le N/2\) of \(N\) i.i.d. votes are correct; by Hoeffding,

\[\Pr[\text{majority wrong}] \le \exp\!\big(-2N(p-\tfrac12)^2\big) \xrightarrow{N\to\infty} 0.\]

This is the Condorcet jury theorem. The catch is independence: correlated errors (the model is confidently wrong the same way) or a wrong modal answer put a floor under the error that more samples cannot break — the empirical plateau of majority voting. Universal Self-Consistency extends voting to free-form outputs by letting the LLM select the most consistent sample.

Best-of-\(N\) / rejection sampling. Sample \(N\), return \(y^\star=\arg\max_i R(x,y_i)\). With a perfect verifier, success equals coverage,

\[\Pr[\text{success}] = 1-(1-p)^N,\]

geometric in \(N\); with an imperfect verifier the ceiling is bounded by verifier accuracy — the verification gap. The unbiased coverage estimator is pass@\(k\): with \(c\) correct of \(n\), \(\text{pass@}k=\mathbb{E}\big[1-\binom{n-c}{k}/\binom{n}{k}\big]\).

7.2 Verifier-guided search

Selection needs a scorer. Two kinds:

• Outcome Reward Model (ORM) — scores the whole solution \(r(x,y)\); rerank best-of-\(N\). Origin: Training Verifiers (GSM8K), which showed verification scales better with data than pure fine-tuning.
• Process Reward Model (PRM) — scores each step \(r(x,y_{1:t})\); aggregate by product / min / last-step. Let's Verify Step by Step showed process supervision beats outcome supervision on MATH (and released PRM800K).

Auto-labeling the process (Math-Shepherd). A step's value is the fraction of Monte-Carlo rollouts from it that reach the correct answer:

\[y_{s_i} = \frac{\#\{\text{rollouts from } s_i \text{ reaching correct answer}\}}{N_{\text{rollouts}}},\]

(hard label = \(\mathbb{1}[\exists\) correct\(]\)), removing human step annotation. OVM trains an outcome-supervised value model \(\approx \Pr(\text{final correct}\mid \text{prefix})\) enabling step-level beam search with no step labels.

Search harnesses. Step-level beam / lookahead expands \(M\) candidates per step and keeps the top-\(b\) by the verifier. Tree search generalizes this: Tree-of-Thoughts (BFS/DFS with self-evaluation; Game-of-24 4%→74%), LATS (MCTS with UCT selection \(Q(s,a)+c\sqrt{\ln N(s)/N(s,a)}\) + reflection), and ★ rStar-Math (MCTS + code-verified steps + a process preference model; Qwen2.5-Math-7B 58.8%→90.0% on MATH, AIME 53.3%, surpassing o1-preview with a 7B model).

8. Test-Time Compute — Reasoning-Model Scaling, RL & Theory

8.1 Compute-optimal test-time scaling

★ Snell et al., "Scaling LLM Test-Time Compute Optimally…" frames the choice of test-time strategy \(\theta\) (parallel breadth vs sequential revision, search depth) as a per-prompt, per-budget optimization:

\[\theta^\star_q(N) = \arg\max_\theta\ \mathbb{E}_{y\sim p_\theta(\cdot\mid q, N)}\big[\mathbb{1}(y\text{ correct})\big].\]

Key finding — the optimum is difficulty-dependent: easy/medium questions favor sequential revision; hard questions favor parallel search/breadth. Optimal allocation uses \(>4\times\) less compute than a best-of-\(N\) baseline, and at matched FLOPs a small model + optimal test-time scaling can beat a \(\sim14\times\) larger model — except on the hardest problems, where pretraining scale still wins.

8.2 Repeated-sampling scaling laws

★ Large Language Monkeys: coverage (fraction solved by any of \(k\) samples) rises smoothly with \(k\), empirically an exponentiated power law,

\[c(k) = 1-(1-\bar{p})^k, \qquad -\log c(k) \approx a\,k^{-b},\]

log-linear over ~4 orders of magnitude (DeepSeek-Coder-V2 on SWE-bench Lite: 15.9%→56% from 1→250 samples). Crucial caveat: coverage keeps rising but selection saturates without an automatic verifier — majority-vote/reward selection plateaus after a few hundred samples. Wu et al., "compute-optimal inference" confirms a smaller model + advanced inference algorithm can dominate a larger model at equal FLOPs.

8.3 Reasoning-model (long-CoT) scaling

RL-trained reasoning models internalize long chains, so test-time compute becomes "think longer." DeepSeek-R1 shows pure-RL (GRPO, group-normalized advantage \(\hat{A}_i=(r_i-\text{mean}(r))/\text{std}(r)\) over sampled groups) elicits emergent long-CoT and self-verification; response length grows over training. ★ s1: Simple test-time scaling controls the thinking budget directly by budget forcing: suppress the end-of-thinking token and append "Wait" to extend reasoning (the model rechecks), or force-close to cap it — yielding a positive accuracy-vs-thinking-tokens slope (s1-32B exceeds o1-preview by up to 27% on competition math).

Sequential vs parallel (the core tradeoff). Sequential scaling (one longer CoT / iterative revision) is more token-efficient and wins on easier problems, but eventually flattens or degrades. Parallel scaling (best-of-\(N\) / search) needs a good selector but keeps lifting coverage on hard problems. Compute-optimal systems mix the two by difficulty (§8.1).

8.4 Test-time RL — optimizing at inference with a self-supervised reward

★ TTRL (Test-Time Reinforcement Learning) runs RL on unlabeled test data using a majority-vote pseudo-reward:

\[r_i = \mathbb{1}\big[y_i = \hat{y}_{\text{majority}}\big], \qquad \hat{y}_{\text{majority}} = \text{mode}\{y_1,\dots,y_N\},\]

then optimizes with GRPO/PPO — the model reinforces its own consensus answers, bootstrapping from pretrained priors with no labels (Qwen2.5-Math-7B: AIME'24 pass@1 ≈ +159%). This is the fusion of test-time compute (§7) and self-improvement: self-consistency provides the reward that a gradient step then folds into the weights at test time. Self-Verification supplies a related backward-checking signal for selection.

8.5 Test-time scaling for diffusion / generation

★ Inference-Time Scaling for Diffusion Models (Ma et al.) reframes extra generation compute as a search over initial/injected noises rather than more denoising steps, along two axes — a verifier (CLIP/aesthetic/reward or self-supervised feature scorer) and a search algorithm (random / zero-order local / search-over-paths). Best-of-\(N\) over noises with a verifier is the simplest instance; quality scales with search compute well past the denoising-step plateau — the diffusion analogue of best-of-\(N\) reasoning.

Key

Test-time compute is search under a scorer. Three levers: how many samples (coverage \(1-(1-p)^N\)), how you search (parallel breadth vs sequential depth, by difficulty), and how you select (majority vote — bounded by \(\exp(-2N(p-\tfrac12)^2)\) but floored by correlated error; or a verifier — bounded by its accuracy). The verifier is the ceiling: perfect verification turns compute into monotone accuracy, weak verification plateaus. TTRL closes the loop by turning the vote into a training reward at inference.

9. Test-Time Adaptation (TTA)

Adapt a pretrained model on unlabeled test data at inference — no source data, no labels — usually via a few backward steps. (Distinguish from source-free DA, which is offline over the whole target set, and from TTT §10, which needs a self-supervised branch baked in at training.)

9.1 Entropy minimization — TENT

Freeze all weights, recompute BatchNorm statistics on the test batch, and update only the BN affine parameters \(\{\gamma,\beta\}\) by minimizing prediction entropy online:

\[\min_{\{\gamma,\beta\}}\ H(\hat{y}) = -\sum_{c=1}^{K} p_c \log p_c, \qquad \hat{y}=\text{softmax}(f(x)).\]

Why entropy, and why it can collapse. Low entropy = confident prediction; under covariate shift, sharpening decisions along the model's own high-density directions recovers accuracy. But the global minimizer of \(H\) is the trivial one: predict a single class with probability 1 for every input (\(H=0\), accuracy destroyed). The gradient w.r.t. logits \(z\), \(\partial H/\partial z_j = p_j\big(\sum_k p_k \log p_k - \log p_j\big) = -p_j(\log p_j + H)\), keeps pushing mass onto the currently-largest logit — so unconstrained entropy minimization drifts toward class collapse. Every robust TTA method below adds a safeguard against exactly this.

9.2 Marginal entropy & information maximization

• MEMO — single-image adaptation: minimize the entropy of the prediction marginalized over augmentations, forcing confidence and augmentation-invariance:

\[\bar{p}(y\mid x)=\tfrac{1}{B}\sum_{i=1}^{B} p_\theta\big(y\mid a_i(x)\big), \qquad \mathcal{L}(\theta;x)=H\big(\bar{p}(\cdot\mid x)\big).\]

• SHOT — freeze the source classifier head, adapt the encoder by information maximization (entropy min + diversity) plus pseudo-labels:

\[\mathcal{L}_{\text{IM}} = \underbrace{\mathbb{E}_x\!\big[H(p)\big]}_{\text{sharpen each}} - \underbrace{H(\bar{p})}_{\text{diversify overall}}, \qquad \bar{p}=\mathbb{E}_x[p].\]

The diversity term \(-H(\bar p)\) is exactly the safeguard from §9.1: maximizing \(I(X;\hat Y)=H(\bar p)-\mathbb{E}[H(p)]\) (mutual information) prevents the all-one-class collapse. Pseudo-labels from k-means centroids on target features add a self-supervised cross-entropy.

9.3 Statistic-only and continual/robust TTA

• BN-statistic adaptation — no gradients at all: blend source and test BN stats \(\bar{\mu}=\tfrac{n}{n+N}\mu_s+\tfrac{N}{n+N}\mu_t\) (and likewise \(\bar\sigma^2\)); "prediction-time BN" is the \(n=0\) case.
• CoTTA — continual streams: a weight-averaged (EMA) teacher \(\theta'_t=\alpha\theta'_{t-1}+(1-\alpha)\theta_t\) produces augmentation-averaged pseudo-labels, and stochastic weight restore \(\theta_t\leftarrow M\odot\theta_0+(1-M)\odot\theta_t\) (\(M\sim\text{Bernoulli}(p)\)) prevents error accumulation/forgetting.
• EATA — sample-selective entropy min (skip high-entropy, redundant samples) + a Fisher anti-forgetting regularizer \(\mathcal{L}_{\text{reg}}=\sum_i\omega_i(\theta_i-\theta_i^0)^2\).
• ★ SAR — reliable + sharpness-aware entropy min for noisy/mixed/tiny batches: \(\min_\theta\max_{\lVert\epsilon\rVert\le\rho}\sum_{x\in S}H(\hat{y}(x);\theta+\epsilon)\), seeking a flat minimum insensitive to noisy gradients, with a reset if the loss collapses. NOTE handles non-i.i.d. streams via instance-aware BN + a class-balanced reservoir.

9.4 VLM / CLIP test-time adaptation

• TPT — for a single test image, optimize only the text prompt by minimizing entropy of the CLIP prediction marginalized over the confident augmented views (lowest-entropy \(\rho\%\)): \(\min_p H(\tilde{p}_p)\), encoders frozen.
• ★ C-TPT — adds a text-feature-dispersion term (spread class embeddings) to fix TPT's miscalibration; DiffTPT uses diffusion-generated augmentations; ★ TDA is training-free — a positive/negative key–value cache of high-confidence test features acts as a dynamic adapter (no backprop).

Key

TTA is a few unlabeled gradient steps against a self-generated objective — usually entropy. The entire literature is a fight against one failure mode: entropy minimization's trivial all-one-class solution. The fixes recur — diversity / marginal-entropy (SHOT, MEMO), reliable sample selection (EATA, SAR), flat minima (SAR), and anchoring to source (CoTTA restore, EATA Fisher). Update as little as possible (BN affine ⊂ encoder ⊂ all) and you stay safe.

10. Test-Time Training (TTT)

TTT bakes a self-supervised auxiliary task into training so it can be used at test time to adapt on the (unlabeled) test input. A Y-shaped network shares a feature trunk \(f_{\theta_e}\) between a main head \(g_{\theta_m}\) and an SSL head \(h_{\theta_s}\).

10.1 The joint objective and the test-time step (original TTT)

Train both tasks jointly:

\[\min_{\theta_e,\theta_m,\theta_s}\ \frac1N\sum_i\Big[\ell_m\big(g_{\theta_m}(f_{\theta_e}(x_i)),y_i\big) + \ell_s\big(h_{\theta_s}(f_{\theta_e}(x_i)),y_i^{s}\big)\Big].\]

At test time, for input \(x\) (no label), take gradient step(s) on the SSL loss only, updating the shared trunk, then predict:

\[\theta_e^\star = \arg\min_{\theta_e}\ \ell_s\big(h_{\theta_s}(f_{\theta_e}(x)),y^s\big), \qquad \hat{y} = g_{\theta_m}\big(f_{\theta_e^\star}(x)\big).\]

The original SSL task is 4-way rotation prediction; TTT-MAE uses masked-autoencoder reconstruction (denser signal, bigger gains); TTT++ (NeurIPS 2021; no arXiv) uses contrastive SSL plus feature-distribution alignment \(\lVert\mu_s-\mu_t\rVert^2+\lVert\Sigma_s-\Sigma_t\rVert_F^2\) to keep test features near training statistics.

10.2 Why TTT helps — the gradient-alignment argument

After one SSL step \(\theta' = \theta - \eta\nabla_\theta\ell_s\), the (unavailable) main loss changes to first order as

\[\ell_m(\theta') \approx \ell_m(\theta) - \eta\,\big\langle \nabla_\theta \ell_m,\ \nabla_\theta \ell_s \big\rangle.\]

So TTT reduces the main loss iff the SSL and main gradients are positively aligned, \(\langle\nabla\ell_m,\nabla\ell_s\rangle>0\), under the shifted distribution. This is the whole theory: pick an SSL task whose gradient correlates with the task gradient (rotation, masking, contrastive all encode useful image structure); TTT++ shows the correlation breaks without feature alignment, which is why naive SSL-TTT can fail. Recent work (TTT provably improves in-context learners) formalizes the risk reduction.

10.3 Single-sample vs online, and the risks

• Single-sample (standard): reset to trained weights per test point, a few steps, predict, discard — no cross-sample state.
• Online: carry \(\theta_e^{(t)}=\theta_e^{(t-1)}-\eta\nabla\ell_s(x_t)\) across a correlated stream — stronger under smooth shift, but drifts under non-stationarity.
• Knobs & risks: adapt the feature extractor, not the decision head (safer); too many steps / large \(\eta\) overfit the SSL task and hurt accuracy; per-sample optimization multiplies inference cost.

10.4 ★ TTT for reasoning — ARC test-time training

For each ARC task (a few input→output demos), fine-tune a per-task LoRA adapter on synthetic data built from that task's own demonstrations, then predict the query — adapter discarded afterward:

\[\Delta_\tau^\star = \arg\min_\Delta \sum_{(x,y)\in\text{LOO-aug}(\tau)} \ell_{\text{LM}}\big(y\mid x;\ \theta_{\text{LM}}+\Delta\big).\]

Leave-one-out augmentation holds out one demo pair as the query and adds invariance transforms (rotations, reflections, color permutations) to expand the tiny per-task set; augmented inference + hierarchical voting then decode. Result: 53% (8B) on ARC public eval, 61.9% combined with program synthesis — a striking case of test-time training (real gradient updates per task) beating pure in-context inference on abstraction.

11. TTT as Sequence Modeling — TTT Layers & Test-Time Memory

The deepest form of the idea: make the model's hidden state itself a model that learns at test time, one gradient step per token.

11.1 ★ TTT layers (TTT-Linear / TTT-MLP)

Replace the RNN's fixed state-update with an actual self-supervised gradient step. The hidden state is a weight matrix \(W_t\); each token updates it:

\[W_t = W_{t-1} - \eta\,\nabla \ell\big(W_{t-1};\, x_t\big), \qquad \ell(W;x_t) = \big\lVert f(W;\theta_K x_t) - \theta_V x_t \big\rVert^2,\]

with learnable low-rank projections (a "training view" \(\theta_K x_t\), "label view" \(\theta_V x_t\), "test view" \(\theta_Q x_t\)), and output \(z_t = f(W_t;\theta_Q x_t)\). TTT-Linear takes \(f(W;x)=Wx\); TTT-MLP takes a 2-layer MLP.

The forward pass literally is learning. For \(f(W;x)=Wx\) and squared loss, the update is the delta rule \(W_t=W_{t-1}-\eta(W_{t-1}\theta_K x_t-\theta_V x_t)(\theta_K x_t)^\top\) — the network's inner loop runs online SGD on a reconstruction task. This is the general case of which linear attention / fast weights are the degenerate, lossless-additive instance (state updated by an outer product, no gradient). Consequences: linear complexity with a fixed-size state (like Mamba), but because the state is a learner rather than a fixed-capacity compression, quality keeps improving past long (>16k-token) contexts. A "mini-batch TTT" dual form makes the per-token step hardware-efficient.

11.2 ★ Test-time memory: Titans & ATLAS

Generalize the inner loop to a long-term neural memory updated at test time with momentum and forgetting:

\[M_t = (1-\alpha_t)M_{t-1} + S_t, \qquad S_t = \eta_t S_{t-1} - \theta_t\,\nabla\ell(M_{t-1};x_t),\]

where the gradient is a "surprise" signal, \(S_t\) is momentum, and the gate \(\alpha_t\) is adaptive forgetting (weight decay). Titans learns what to memorize on the fly; ATLAS optimizes memory over a sliding window of context (not just the current token) with higher-order updates, reaching +80% accuracy at 10M-token context on BABILong. These are RNN-with-a-learner architectures where "test-time training" is the core computation, not an add-on.

11.3 Meta-learning framing (bi-level)

TTT is the inner loop of a two-timescale optimization; MAML is the training-time analogue:

\[\underbrace{\min_\theta \sum_\tau \ell_\tau^{\text{outer}}(\theta_\tau^\star)}_{\text{meta / train time}} \quad\text{s.t.}\quad \underbrace{\theta_\tau^\star = \theta - \eta\,\nabla_\theta\,\ell_\tau^{\text{inner}}(\theta)}_{\text{adaptation / test time}}.\]

MAML's inner loss is a supervised support-set loss (learn an init that adapts fast); TTT's inner loss is a self-supervised loss on the test input; TTT layers make the outer loop ordinary next-token pretraining and the inner loop the per-token SSL step — "learning to learn at test time."

12. Test-Time Optimization Without Weight Updates

Optimization at inference need not touch weights — it can run in text or latent space.

• ★ Test-Time Preference Optimization (TPO) — align to a reward model at inference with no parameter updates. Sample candidates, score them with a reward model, convert the scores into textual critiques ("textual rewards"), and iterate — a textual gradient in response space:

\[y^{(k+1)} = \text{Model}\big(x,\ y^{(k)},\ \text{critique}(y^{(k)})\big), \qquad \text{critique}=\text{LLM}\big(\{y_i, r(y_i)\}\big).\]

An unaligned Llama-3.1-70B-SFT surpasses its RLHF-aligned counterpart after a few inference steps. This is the training-free analogue of DPO: the same preference signal, applied to tokens instead of weights.

• Latent / prompt test-time optimization — TPT (§9.4) optimizes continuous prompt vectors per test image; latent-variable optimization (optimize the initial noise / latent under a verifier) underlies diffusion test-time search (§8.5). In-context learning itself is test-time optimization with zero steps — the forward pass implements the learning algorithm (see the ICL-as-gradient-descent result in the companion prompt/context/harness sheet).

Key

"Test-time training" and "test-time optimization" form a ladder of increasing intrusiveness: no update (ICL, TPO textual gradients, guidance) → inputs/activations (TPT, steering) → a few params (TENT BN, ARC LoRA) → a state-model (TTT layers, Titans) → all params (MEMO, online TTT). Move only as far down the ladder as your signal reliably supports.

13. Unifying View & Practical Guidance

One skeleton. Every method here instantiates: choose what to optimize \(\phi\), drive it with a signal \(s\), spend parallel or sequential compute, subject to a constraint that bounds drift.

\[\phi^\star = \arg\min_\phi\ \mathcal{L}_{\text{signal}}(\phi; x) \quad \text{s.t.}\quad \text{drift}(\phi,\phi_0)\le \rho,\]

where \(\mathcal{L}_{\text{signal}}\) is a contrast (decoding), a guidance loss (diffusion), negative agreement (self-consistency/TTRL), entropy (TTA), self-supervision (TTT), or a preference critique (TPO); and the constraint is a plausibility mask, a KL/EMA anchor, a Fisher penalty, "update BN only," or "reset per sample." Read that way, decoding, guidance, search, TTA, TTT, and TPO are one family at different \(\phi\).

Choosing (rough guide).

Pitfalls. Entropy-min collapse (§9.1); majority-vote plateau under correlated error (§7.1); verifier gap caps all search/self-reward (§7–8); TTT overfitting/drift with too many steps or online non-stationarity (§10.3); merging interference outside the linear basin (§4); CFG scale off-by-one across conventions (§6). And always weigh the compute multiplier — most of these trade FLOPs (sometimes \(10\text{–}1000\times\)) for accuracy.

Appendix A: Twenty-Five Things to Memorize

1. Two families: training-free inference vs test-time optimization/training.
2. Three axes: what's optimized, what signal, parallel vs sequential compute.
3. Decoding contrasts: CD, DoLa, CAD, CFG-LM all extrapolate along a log-prob difference.
4. MBR: decode the consensus (max expected utility), not the mode.
5. Steering: \(h\leftarrow h+\alpha v\) — the training-free dual of fine-tuning (RepE, CAA, ITI).
6. Task vector \(\tau=\theta_{\text{ft}}-\theta_{\text{pre}}\); merge by \(\theta_0+\sum_i\lambda_i\tau_i\) (Task Arithmetic).
7. TIES/DARE suppress merge interference (trim, elect-sign, drop-and-rescale).
8. Speculative decoding is lossless: accept w.p. \(\min(1,p/q)\), resample residual \((p-q)_+\).
9. Proof: \(\min(p,q)+(p-q)_+=p\) → emitted token \(\sim p\) exactly.
10. Expected tokens per block \(=\tfrac{1-\alpha^{\gamma+1}}{1-\alpha}\) (EAGLE/Medusa).
11. CFG: classifier-free guidance = implicit classifier \(\nabla\log p(y\mid x)=\nabla\log p(x\mid y)-\nabla\log p(x)\).
12. Self-consistency error \(\le\exp(-2N(p-\tfrac12)^2)\); floored by correlated error.
13. Best-of-\(N\) success \(=1-(1-p)^N\) with a perfect verifier; else verifier-bounded.
14. PRM > ORM > majority vote, gap widens with \(N\) (Let's Verify).
15. Math-Shepherd: step value = fraction of rollouts reaching the right answer.
16. Compute-optimal test-time scaling is difficulty-dependent: sequential for easy, parallel for hard (Snell).
17. Coverage power law \(-\log c(k)\approx a k^{-b}\) (Large Language Monkeys); selection saturates without a verifier.
18. ★ s1 budget forcing: append "Wait" to extend thinking.
19. ★ TTRL: RL at test time with a majority-vote pseudo-reward.
20. ★ Diffusion test-time scaling = search over noises with a verifier (Ma et al.).
21. TENT: minimize prediction entropy over BN affine params; risks class collapse.
22. Collapse fix = diversity/mutual-info (SHOT), reliable selection + SAM (SAR), anchoring (CoTTA).
23. TTT helps iff \(\langle\nabla\ell_m,\nabla\ell_s\rangle>0\) (SSL–main gradient alignment).
24. ★ TTT layers: hidden state is a model; \(W_t=W_{t-1}-\eta\nabla\ell(W_{t-1};x_t)\) per token (linear attention is the no-gradient special case).
25. ★ TPO: align at inference via textual gradients, zero weight updates — the training-free DPO.

Appendix B: Decision Guide — "Which Test-Time Method?"

1. Want the same outputs, faster? → Speculative decoding (EAGLE/self-spec/lookahead).
2. Want higher quality for free from a frozen LM? → Decoding contrast (DoLa/CAD) or, for images, guidance (CFG/PAG).
3. Want to change behavior with no data/training? → Activation steering (CAA/RepE).
4. Have multiple finetunes? → Merge (Task Arithmetic → TIES+DARE).
5. Hard problem + automatic checker? → Best-of-\(N\) / PRM search; allocate by difficulty (Snell).
6. Hard problem + no checker? → Self-consistency, or ★ TTRL to fold the vote into weights.
7. Reasoning model, want to dial effort? → ★ budget forcing (s1).
8. Distribution shift, unlabeled stream? → TTA: TENT → ★ SAR/EATA; update BN affine first.
9. Per-instance/task shift + an SSL task? → TTT (MAE-based); abstraction → ★ ARC per-task LoRA.
10. Long-context / streaming? → ★ TTT layers / Titans / ATLAS.
11. Align/steer outputs at inference, no retrain? → ★ TPO; VLM zero-shot → TPT/TDA.

Appendix C: Formula Sheet

• Contrast decoding: \(\text{score}=\log p_A-\lambda\log p_B\) over a plausibility-masked vocab.
• Guidance: \(\tilde{\varepsilon}=\varepsilon_\varnothing+s(\varepsilon_c-\varepsilon_\varnothing)\); classifier: \(\tilde{\varepsilon}=\varepsilon-s\sigma_t\nabla_{x_t}\log p_\phi(y\mid x_t)\).
• Speculative accept: \(\min(1,p(x)/q(x))\); residual \((p-q)_+/\sum(p-q)_+\); tokens/block \(\tfrac{1-\alpha^{\gamma+1}}{1-\alpha}\).
• Self-consistency: \(\hat a=\arg\max_a\sum_i\mathbb{1}[a_i=a]\); error \(\le e^{-2N(p-1/2)^2}\).
• Best-of-\(N\): \(1-(1-p)^N\); pass@\(k=\mathbb{E}[1-\binom{n-c}{k}/\binom{n}{k}]\).
• Coverage law: \(-\log c(k)\approx a k^{-b}\).
• Compute-optimal: \(\theta^\star_q(N)=\arg\max_\theta \mathbb{E}[\mathbb{1}(\text{correct})]\).
• Entropy TTA: \(\min H(\hat y)=-\sum_c p_c\log p_c\); info-max \(I=H(\bar p)-\mathbb{E}[H(p)]\).
• TTT step: \(\theta_e\leftarrow\theta_e-\eta\nabla_{\theta_e}\ell_s(x)\); helps iff \(\langle\nabla\ell_m,\nabla\ell_s\rangle>0\).
• TTT layer: \(W_t=W_{t-1}-\eta\nabla\ell(W_{t-1};x_t)\), output \(f(W_t;\theta_Q x_t)\).
• Titans memory: \(M_t=(1-\alpha_t)M_{t-1}+S_t,\ S_t=\eta_t S_{t-1}-\theta_t\nabla\ell(M_{t-1};x_t)\).
• Merging: \(\theta=\theta_0+\sum_i\lambda_i\tau_i\), \(\tau_i=\theta_i-\theta_0\).
• TPO: \(y^{(k+1)}=\text{Model}(x,y^{(k)},\text{critique}(y^{(k)}))\).

Appendix D: Year-by-Year Milestones

• 2020: TTT (rotation SSL); TENT; BN-adapt; SHOT.
• 2021: Classifier guidance; Fisher merging; TTT++ (feature alignment).
• 2022: Self-Consistency; CFG; Model Soups; Contrastive Decoding; Task Arithmetic; MEMO; CoTTA; TPT; TTT-MAE; GSM8K verifiers; spec decoding.
• 2023: DoLa; CAD; RepE/ITI; TIES/DARE; Let's Verify; Math-Shepherd; ToT; SAR; universal guidance.
• 2024: ★ Snell compute-optimal; ★ Large Language Monkeys; Medusa/EAGLE/lookahead; PAG; ★ TTT layers; ★ ARC-TTT; TDA.
• 2025: ★ DeepSeek-R1; ★ s1; ★ rStar-Math; ★ TTRL; ★ diffusion test-time scaling; ★ Titans/ATLAS; ★ TPO; TTT provable ICL.
• 2026: test-time compute a first-class scaling axis alongside pretraining; training-free inference (steering, merging, guidance, spec-decoding) standard production tooling; TTT-as-memory architectures maturing for long context; the binding constraint everywhere remains the verifier / signal.