The universal AI memory platform
More AI.
Same GPUs.
Yantrion shrinks the memory AI needs to do its work, and lets the GPU use it without unpacking it first. The result: faster answers, more users and far longer context, on the GPUs you already own.
Every number below was measured on real hardware, side by side with the standard stack, and checked for quality. None of it is projected.
- More users at once
- 2.54×
- same GPU memory, Qwen3.8
- Tokens of context
- 1,000,000+
- the standard stack stops at 307,571, Kimi-K3
- More tokens per second
- 2.1×
- 1,926 vs 905 at 128 users, AMD MI350X
- Agents resumed with memory intact
- 150/150
- the standard stack: 0 of 150
Models
Kimi-K3, Qwen3.8, DeepSeek-V4, GLM-5.3, GPT-OSS, Nemotron 3
Hardware
AMD Instinct MI350X, NVIDIA H200
Runs on
SGLang, vLLM, TensorRT-LLM, NVIDIA Dynamo
What we do
From algorithm to production performance.
- 1Design
Rethink the math
We find the parts of an AI model's memory that can be made smaller without changing the model's answers in our quality checks, and leave the parts that must stay exact untouched.
- 2Build
Write it for the GPU
Our GPU code works directly on the compressed memory, so nothing has to be unpacked first. Tuned for AMD and NVIDIA, and checked against a trusted reference.
- 3Ship
Plug it in where AI runs
It fits into the serving software teams already use: SGLang, vLLM, TensorRT-LLM and NVIDIA Dynamo. No retraining, no model changes.
Why most memory savings disappear
Illustration. Box width shows how much memory each step takes.
The usual way
Memory is shrunk, then unpacked back to full size before the GPU can use it. Most of the savings disappear.
Yantrion
The GPU works on the smaller memory directly. Nothing is unpacked, and the answers stay the same.
One platform
Every attention design. One platform.
| Model design | Models measured | Memory made smaller |
|---|---|---|
| Latent attentionMLA | Kimi-K3, DeepSeek-V4 | up to 2.3× smaller |
| Grouped attentionGQA | Qwen3.8, GPT-OSS, Nemotron 3 | up to 2.9× smaller |
| Sparse attentionDSA | GLM-5.3 | 1.9× smaller |
| Classic attentionMHA / MQA | The same GPU code applies | Supported |
Some parts of a model's memory must stay exact, so we leave them untouched and shrink only what grows as conversations get longer. Savings are measured against each model's own standard memory format.
“Moving from one model family to a completely different one took days, not months. Only a small model-specific piece changed; everything else carried over.”
Proof, not promises
Not a paper. A measured system.
Speed
Faster, and faster still with longer context.
Yantrion produces more tokens per second at every load we tested, and the lead grows as conversations get longer.
At 128 users, the standard stack handled only 54 of them. Yantrion delivered 1,926 tokens per second, more than double.
Our standard-stack numbers land within about 5% of independent published results for the same model on AMD MI355X (Wafer, July 2026), so this is a fair comparison. No speed tricks like speculative decoding are included.
Kimi-K3 on AMD Instinct MI350X with SGLang, generating 1,024 tokens per request. Every run passed our quality checks before it counted.
† At 128 users the standard stack handled only 54 of the 128 (striped). Yantrion's 128-user number is a conservative floor.
Show the numbers
| Users | Standard stack (tokens/s) | Yantrion (tokens/s) | Gain |
|---|---|---|---|
| 4 | 159 | 255 | 1.60× |
| 32 | 722 | 1,033 | 1.43× |
| 128 | 905 (handled only 54 of 128) | 1,926 | ≥2.1× |
At 28K tokens the lighter bar shows the measured range.
Show the numbers
| Context and users | Speed vs standard stack |
|---|---|
| 28K tokens, 17 users | 1.44–1.62× |
| 128K tokens, 4 users | 1.72× |
| 256K tokens, 4 users | 2.32× |
More users
More people on the same GPUs.
Smaller memory leaves room for more. Qwen3.8 served 193 users at once where the standard stack fit 76, and DeepSeek-V4-Flash held 53% more tokens in exactly the same GPU memory.
Every comparison uses the model's own standard memory format, never an easy baseline.
Show the numbers
| Model | What we counted | Standard stack | Yantrion | Gain |
|---|---|---|---|---|
| Qwen3.8 | users at once | 76 | 193 | 2.54× |
| Kimi-K3 | users at once | 16 | 32 | 2.00× |
| DeepSeek-V4-Flash | tokens in memory | 1,964,288 | 3,013,376 | +53% |
Long context
Context the standard stack can't hold.
Kimi-K3 read a single prompt of 1,048,576 tokens, roughly ten novels' worth of text, with 128 of 128 identical answers. On the same setup, the standard stack stops at 307,571 tokens.
1,048,576
tokens in a single prompt, on Kimi-K3
Every quality check passed. On the same setup, the standard stack stops at 307,571 tokens.
The standard stack stops at 307,571 tokens. Yantrion handled a 1,048,576-token prompt.
Long context, measured
- 1,048,576 tokens in one prompt on Kimi-K3
- 2.32× faster than the standard stack at 256K tokens
- A 1M-token context on four H200s for Nemotron 3 Ultra, half the usual eight
AI agents
AI agents that keep their memory.
Agents pause, wait and come back. After we cycled an entire fleet, all 150 agents resumed with their memory intact. The best standard setup that would run kept none, so every agent had to start over.
When we grew the fleet from 150 to 300 agents, not a single request failed. It slowed down gracefully instead of breaking.
Show the numbers
| Setup | Agents resumed with memory intact |
|---|---|
| Yantrion | 150 of 150 |
| Best standard setup that would run | 0 of 150 |
Show the numbers
| Agents running | Resume from memory | Start over |
|---|---|---|
| 150 | 12.7 s | 28.5 s |
| 96 | 1.7 s | 33.2 s |
0
Failed requests
while growing from 150 to 300 agents
580 vs 116
Saved agent sessions per machine
Nemotron 3 Ultra, with and without Yantrion
Quality
Quality comes first.
A result only counts if the model gives the same answers. Our own checks have thrown out results that looked good but didn't hold up, and we're proud of that.
See every number, what it was compared to, and the setup it ran on.
128/128
Same answers on agent tasks
Multi-step agent conversations matched word for word. No partial credit.
Pass
Long-document recall
Needle-in-a-haystack tests: the hidden fact is found.
15/16
Math reasoning
GSM8K grade-school math problems.
195
Runs on record
Every test run saved and fingerprinted.
Smart memory
Nothing gets thrown away.
Active
Always on
In GPU memory, ready to use.
Shrink and set aside when memory is tightBack when neededSet aside
1.7–3.0× smaller
Shrunk and parked when memory gets tight, and back when it's needed. Nothing is thrown away.
Seal and save, or send to another machineRestored exactly as it wasSaved
1.9–2.1× smaller; 3.7× smaller to send
Sealed and stored, ready to move to another machine and come back exactly as it was.
Runs on your stack
Works with the software you already use.
SGLang
High-speed open-source serving engine
Supported and qualified
vLLM
Widely used open-source serving engine
Supported and qualified
TensorRT-LLM
NVIDIA's high-performance serving engine
Supported and qualified
NVIDIA Dynamo
AI serving at data-center scale
Supported and qualified
NIXL
Fast transfer of AI memory between GPUs and machines
Supported and qualified
LMCache
Stores and reuses AI memory
Supported and qualified
What it's worth
Same hardware. Far more AI.
2.54×
More users on the machines you have
Up to 2.54× more people served at once on the same GPU memory, with the same quality.
4, not 8
Fewer GPUs for the same job
Nemotron 3 Ultra served a million-token context on four NVIDIA H200s with compact 4-bit model weights, instead of the usual eight. That's the shape of the savings.
1M tokens
Things you couldn't do before
Million-token prompts where the standard stack stops at 307,571, and agent fleets that keep their memory.
Honest by default
How we measure, and where we're still growing.
- Every comparison is against the model's own standard setup, never an easy baseline.
- Our biggest speed wins come when many people use a model at once. For a single user, speed isn't yet where our many-user results are.
- No speed tricks like speculative decoding are counted. Those would come on top.
- If every agent is paused and resumed at the very same moment, 54 of 150 come back with memory intact today.
Where we're going
Memory is the next frontier of AI.
- Now
More AI on the GPUs you own
Faster answers, up to 2.5× more users, million-token context, and agent fleets that keep their memory. One integration, no retraining, running today on AMD and NVIDIA.
Flat per-GPU license
- Next
Agents that never forget
Pause an AI agent, move it to another machine, and pick up exactly where it left off, even days later. Memory that belongs to the agent, not to whichever server it happened to run on.
Early-access program
- Beyond
Memory without walls
AI memory that flows freely across GPUs, machines and data centers, the way data flows across the internet today. When memory stops being the limit, AI can think longer, remember more and reach further.
Exploring with design partners
How we work
We share our misses, too.
In August, one of our own tests ran 1.48× slower than the standard stack. We said so, found the cause in our GPU code, and fixed it. A month later, the same test runs 1.44–1.62× faster.
Show the numbers
| Month | Compared to the standard stack |
|---|---|
| August 2026 | 1.48× slower |
| September 2026 | 1.44–1.62× faster |
Getting started
See it on your own workload first.
Plug in
Works inside the serving software you already use.
Compare
Side by side with what you run today, on your own workload.
Stay flexible
Your standard setup stays right underneath.
Who we are
Hard problems. Simple, elegant answers.
We love the math, the algorithms and the GPU code underneath AI. Most of all, we love turning something deeply complex into something that simply works, and showing the world what becomes possible when it does.
- 1
Start with the math
Every breakthrough starts on a whiteboard. We look for the simple idea hiding inside a hard problem.
- 2
Bring it to the hardware
Ideas only matter when they run. We write the GPU code ourselves, tuned for AMD and NVIDIA, and check it against a trusted reference.
- 3
Ship it for real
Not a paper, a working system. Measured on real hardware, inside the software teams already use.
- 4
Show the world what's possible
Memory is only the beginning. We keep learning, and we keep taking on the toughest problems in AI and computing.
Simple, elegant, endlessly curious. And we're just getting started.
FAQ
Questions people ask us.
What is KV cache compression, in plain words?
While an AI model works, it keeps notes in GPU memory so it doesn't redo work. That memory, called the KV cache, fills up fast with long documents, many users or AI agents. Compressing it lets the same GPUs serve more people, longer conversations and bigger agent fleets.
How is Yantrion different?
Most approaches shrink the memory but have to unpack it before the GPU can use it, which gives most of the savings back. Yantrion's GPU code works on the compressed memory directly. Nothing is unpacked, and the model's math stays exactly the same.
Does it change the model's answers?
Not in our tests. We only publish results that pass our quality checks: 128 out of 128 identical answers on multi-step agent tasks, long-document recall tests passed, and GSM8K math at 15 of 16.
Which models does it work with?
Every major attention design: latent attention (MLA), grouped attention (GQA), sparse attention (DSA) and classic attention (MHA and MQA). We've measured Kimi-K3, Qwen3.8, DeepSeek-V4, GLM-5.3, GPT-OSS and Nemotron 3.
Which serving software and GPUs does it run on?
SGLang, vLLM, TensorRT-LLM and NVIDIA Dynamo are all supported and qualified, along with NIXL and LMCache for moving and storing AI memory. It runs on AMD Instinct and NVIDIA GPUs, including the AMD Instinct MI350X and NVIDIA H200.
Do we need to retrain or change our model?
No. Yantrion plugs into the serving software you already use. No retraining, no model changes.
How long a context can it handle?
Kimi-K3 handled a single 1,048,576-token prompt on a setup where the standard stack stops at 307,571 tokens. Nemotron 3 Ultra served a 1M-token context on four NVIDIA H200s with compact 4-bit model weights, half the eight it normally runs on.
Are there hidden speed tricks in your numbers?
No. We don't count add-ons like speculative decoding. Those gains would come on top of ours.
Try it on your workload
Your model. Your GPUs. See the difference.
Don't take our word for it. Run Yantrion on your own model and hardware, compare it side by side with what you run today, and decide for yourself. We'd love to show you what's possible.
Write to us and we'll set up a pilot on your stack.