Gate On-Device LLMs on Real Snapdragon — Built on Qualcomm’s GenieX

The wall we hit last time

EdgeGate has always run computer-vision models on real Snapdragon silicon and signed the evidence. LLMs were the stubborn exception. The bespoke path we tried — hand-built Genie context binaries profiled through the cloud — kept hitting structural walls: per-part profiling segfaulted on the shared KV cache, and the quantization ops we needed weren’t supported by the runtime version on the device. So on-device LLM gating sat unshipped, and the design-partner conversations that needed it stalled.

What changed: GenieX

Qualcomm shipped GenieX, a community, on-device GenAI inference runtime for Snapdragon. It runs a HuggingFace GGUF or an AI Hub bundle directly on the device — CLI, Python, or an OpenAI-compatible local server — and exposes per-call metrics: time-to-first-token, decode tokens/sec, and, on its qairt runtime, the compute unit the model actually ran on. It removed our execution wall overnight.

We made a deliberate call to consume GenieX as-is rather than fork it: its public API already surfaces everything the gate needs. GenieX is the execution engine. It is not the thing that tells you your quantized model still refuses the request it’s supposed to refuse.

What EdgeGate adds on top

A runtime answers “how fast.” A gate answers “did the behaviour change.” EdgeGate wraps a GenieX run in the same Behavioral Gate it uses for CV:

• Run the quantized LLM on the device against a published eval set.
• Diff each output against a signed reference oracle — refusals, forbidden actions, task answers — not a byte-compare, a behavioural one.
• Record peak memory and the observed compute placement.
• Emit a single Ed25519-signed evidence bundle; the raw model text never leaves the device (summary-only, NDA-safe).
• Return an exit code your CI branches on — RED fails the build.

The sharpest signal: did it actually hit the NPU?

Quantization precision decides placement. A Q4_0 GGUF or a w4a16 AI Hub bundle lands on the Hexagon NPU; other precisions quietly fall back to CPU or GPU — same output, three times the latency and battery. A benchmark run on a fast machine won’t catch it. Because GenieX reports the device it ran on, EdgeGate can make placement a gate-able signal: fail the build if your LLM stopped running on the NPU. That is the silent-fallback story every edge team knows from CV — now on the LLM tier.

What a real run looks like

This isn’t a mock. On a Snapdragon X Elite we captured a reference baseline, ran a gated GGUF LLM through GenieX, and got:

Everything in the summary is inside the signed payload, so the memory and placement claims are tamper-evident — you can hand the bundle to an assessor and they can verify it without trusting us.

How you run it

The device runner is a slim PyPI package. On any Snapdragon host — including a Windows-on-ARM X Elite box, no ADB needed, because GenieX runs the model in-process:

Point a self-hosted GitHub runner at that host and every pull request that touches your model gets gated on real silicon. Both the Android/ADB genie-bundle path and the GenieX on-device-LLM path are first-class in the dashboard and the CI action.