TL;DR
Researchers fit a mapper that converts one model’s KV cache into another’s. Three of their model pairs fit the calibration data identically well — R²=0.84 — and then retained 94%, 93%, and 37% of downstream benchmark accuracy. Across twelve evaluations, fit quality correlated with actual behaviour at r=−0.20: essentially not at all. Their conclusion is that what matters is where the residual error lands relative to the subspaces attention actually reads, not how large it is. If you validate a quantized edge model with MSE, SQNR, or cosine similarity against an FP reference, that finding is about you.
What the paper was actually trying to do
Cross-Model KV Cache Transfer in LLM Families (arXiv:2608.03893) tackles a serving problem. Production systems switch between differently-sized models in the same family — cascading from small to large, routing by difficulty, escalating mid-conversation. Every switch means re-reading the entire context to rebuild the new model’s KV cache from scratch.
Their proposal: don’t. Fit a per-head ridge regression that maps the source model’s cache into the layout the target expects, and skip most of the prefill. They strip RoPE from the keys before fitting, which makes the mapper position-free and reusable across context lengths — a genuinely clever detail. On six pairs across three families it works well enough to be interesting: Qwen3 14B→32B retains 97.6% of average benchmark accuracy, and they report 3–25× speedups over re-prefill.
That’s the paper’s headline. It isn’t the part worth your time.
The part worth your time
Two of the six pairs collapsed. Ministral 3B→14B retained 44.2%; 8B→14B retained 41.6%. So the authors went looking for what separates a working transfer from a broken one — and the obvious candidate failed.
Three separate transfers fit their calibration data at an identical R² of 0.84. Llama 3.1 8B→70B held 94% of HellaSwag accuracy going small-to-large, and 37% going large-to-small. Ministral 3B→8B, at the same R², held 93% in both directions. Same number on the fit. Wildly different models afterward.
Across twelve pair evaluations they measured it properly. Calibration R² correlated with downstream retention at r=−0.20 — the paper’s words are that it “shows essentially no correlation.” Attention-output cosine similarity did better at r=+0.57, but that is a post-hoc diagnostic: you can only compute it after you’ve already built the thing you wanted to validate.
“Per-pair retention is determined by where the residual error lands relative to the target’s attention-sensitive subspaces, not by its magnitude.”
— arXiv:2608.03893, §6
They made this concrete. For each head, they project the per-token error onto the right singular vectors of the target’s query matrix and weight by the matching singular values. In their framing: “Concentration above 1 means the error concentrates where attention reads, while below 1 means it lands where attention ignores.” Two models can be equally wrong by any norm you like. Only one of them is wrong in a direction the network is looking.
Why this is a quantization result in disguise
Cross-model cache transfer and INT4 quantization look like different problems. Structurally they are the same problem: you have applied a small numerical perturbation to a network’s internals, and you want to know whether it still behaves.
And the industry validates quantization exactly the way this paper just showed doesn’t work. Mean squared error against the FP32 reference. Signal-to-quantization-noise ratio per layer. Cosine similarity of output logits. Every one of those is a magnitude measure. Every one is blind to placement. A team can quantize a model, watch per-layer SQNR stay inside tolerance, ship it, and discover in the field that a class of inputs now resolves differently — because the error, small as it was, landed somewhere the attention pattern depends on.
The paper also shows how badly a calibration-fit metric can mislead once you leave the calibration distribution. On the two failing pairs, the ridge mapper’s R² measured on the evaluation domain was −3.22 and −7.81 — catastrophically negative, from a fit that looked healthy on the 500 calibration sequences it was trained on. Swapping the linear mapper for an MLP rescued one failure from 58.7% to 95.5% accuracy, a 36.8-point recovery, but even then the eval-domain R² stayed below zero. The behaviour recovered before the reconstruction metric did. The metric was never the thing.
If you calibrate a quantizer on a few hundred representative samples and validate against the same distribution, you have built the identical trap.
What this paper is not evidence for
Worth being direct, because the temptation to overclaim here is real: cross-model KV cache transfer is a serving-tier technique. It needs two models resident and a router switching between them. That is a datacenter concern. Nobody is hot-swapping a 14B for a 32B on a handset, and on-device runtimes like Genie compile fixed KV layouts into the context binary — there is no runtime facility to remap another model’s cache into one. EdgeGate does not gate KV cache transfer, and this paper does not say it should.
What it does is validate the methodology argument, from a completely different direction, with numbers we didn’t have to generate ourselves. Nine researchers set out to make prefill cheaper, and the finding they had to publish along the way is that numerical similarity to a reference does not predict behavioural equivalence. That is the entire premise of behavioural gating.
The drift number nobody should skip
One more result, buried in §4.6. They ran multi-turn CoQA — 100 conversations of roughly 15 turns across five domains — and measured how far the mapped model drifted from the target model’s standalone answers. Large-to-small drift was linear at 0.33 percentage points per turn.
The authors are honest that this is too small to cause cascading failure within ten turns. But linear drift doesn’t stop at turn ten. A single-turn benchmark would have scored this transfer as fine, because at turn one it was fine. Whatever you are validating, if it holds state across a session, a single-shot eval is measuring the easiest possible case.
What to take from it
Three things, and none of them require you to care about KV caches.
- A reconstruction metric is not a validation. MSE, SQNR, and cosine similarity tell you how far you moved. They do not tell you whether you moved somewhere that matters.
- Calibration-set agreement is the weakest possible evidence. A fit that looked clean on its calibration data went to R²=−7.81 off-distribution. Validate on inputs the tuning process never saw.
- Behaviour is the only thing worth gating on. Run the task. Compare the answers to a signed reference. Block the build when they diverge. Everything else is a proxy, and this paper is a long, careful demonstration of a proxy failing.
That last point is the one EdgeGate is built around, and it’s why our gates compare on-device outputs against a certified reference rather than against a tolerance band on a distance metric. Quantize a model, change a runtime, bump firmware, move to new silicon — the numbers can stay inside every threshold you set while the model quietly starts answering differently. The only way to know is to ask it the questions that matter and check the answers, on the hardware it actually ships on.
Numerical similarity is not behavioural equivalence.
EdgeGate runs your model on real Snapdragon and Jetson hardware, diffs the behaviour against a signed reference, and blocks the build when it drifts — with tamper-evident evidence for every run.
See what EdgeGate gates →Related
© 2026 EdgeGate. Powered by Qualcomm AI Hub.