Keeps each session on its cache
Requests from one agent session go to the GPU that already holds its KV cache, so old context isn't computed again.
Your agent shouldn't pay for the same tokens twice.
Lumenar is an inference layer for long-running AI agents on open models. It reuses the context your agent keeps re-sending, and keeps the session intact when you switch models.
Built on vLLM and SGLang. One OpenAI-compatible API.
Most of an agent's bill is re-reading.
Every turn, an agent sends its whole history back to the model: the system prompt, the tools, the files it opened, every earlier step. The history grows each turn, and you pay for all of it again.
99.2%
of all tokens in our own agent sessions were input, not output.
88.5%
of those input tokens had been sent before.
Measured on our own coding-agent logs: 17,780 model calls in 1,305 sessions, 1.43B prompt tokens.
Switching models throws the session away.
Teams change models for price, rate limits or quality. But a model's cache is computed from its own weights, and each model has its own tokenizer, chat template and tool-call format. So the new model starts cold: it reads the whole history again, and the plan can get lost on the way. Most teams just avoid switching, and live with the lock-in.
What Lumenar does
Lumenar sits between your agent and your models. We extend open engines; we don't build a new one.
Lays out prompts so they stay cacheable
Stable parts go first and new turns go last. One changed token early in the prompt breaks the cache for everything after it.
Picks a model for each step
Reading files, searching and running tests go to a small model. Planning and debugging go to a large one.
Hands the session over on a switch
The session lives in a log that no single model owns, so a new model can pick it up where the last one stopped.
What happens when you switch
- Freeze. Stop at a clean turn boundary.
- Compact. Fold old turns into the session log, keeping the plan, decisions and open files.
- Render. Write the log in the new model's chat template and tool format.
- Warm up. One prefill fills the new model's cache.
- Check. A short probe confirms the new model knows the plan.
- Go on. The agent continues from the same turn.
What it could save
For one 60-turn example task, our cost model puts the bill at $2.95 with no reuse, $0.47 with a prefix cache, and $0.30 when easy steps also go to a smaller model.
Assumptions: $1 per 1M new input tokens, $0.10 per 1M cached, $3 per 1M output; 12K to 84K tokens per turn; 70% of steps can use a model at one fifth of the price. A pilot measures the real numbers.
Where we are
We're at prototype stage and pre-revenue. Here's exactly what's done and what isn't.
Done
- A market study of 8.56M public posts and issues on inference
- The architecture, and a 12-week plan with go / no-go gates
- A baseline on real agent logs: 88.5% of prompt tokens repeat
- A shipped product: Clutch, a basketball recruiting platform
Not yet
- A working prototype of the session layer. The build starts now.
- Paying customers
- An external benchmark. That's what a pilot is for.
Team
Running long agents on open models?
We're looking for teams and GPU partners for a pilot. We run your real agent workloads, baseline against Lumenar, and judge it on one number: cost per solved task.