agent-platform
One OpenAI-compatible endpoint in front of every model, hosted or running on the desk.
Problem
An agent that is useful is an agent wired to several providers, and each one arrives with its own SDK, its own auth and its own idea of what a streamed response looks like. The local runtimes are worse: the code that talks to Ollama on a laptop is not the code that talks to it from inside a container, so the thing that works while developing is not the thing that ships.
Approach
One Rust binary (`agent-platformd`, Axum on Tokio) serves the multi-agent orchestration API and an OpenAI-compatible /v1 surface from the same process, so any client already written against OpenAI points at it and needs no branch per provider. It began as a FastAPI server and was replaced in place by the Rust one, a route at a time, under ADR 0007. Callers can bring their own key: three headers switch a request onto the caller’s credential and the server spends none of its own quota. Local runtimes are reachable from a container because loopback addresses are rewritten to the host at the boundary rather than in every call site. The UI is a native desktop app in Rust and iced. The server ships no browser UI at all.
Outcome
- Nine upstreams behind a single endpoint, seven hosted providers and two local runtimes, with no provider-specific path in the client.
- A prompt can be answered by a model on the same machine: no key issued, nothing leaving the host, the same code as the hosted route.
- A caller’s own provider key is accepted per request, and the base URL it may be pointed at is allowlisted, https-only and never a raw IP, so BYOK cannot be turned into a way to reach internal services.
- Every provider reports the same capability shape, with a documented fallback chain when one cannot list its models live.
- One Docker image, SQLite on a named volume: it comes back up as it went down.
1 / 3
The migration is the part worth defending. Two requirements were in conflict, to ship embedded with a desktop app and to serve other API clients as a real server, and a third settled it: deploy to the cloud with no desktop client, which rules out one binary carrying both the GUI and the API. The answer was a strangler, not a port: a Rust process in front of the Python one, domains moved across a few at a time, the Python server still answering everything that had not moved. No flag day, and abandonable at any point without losing what had already landed. It ran 5–7 August 2026 and the Python server is now deleted.
Completing it meant overriding a rule the migration itself had set, that a migrated route must be byte-identical to the one it replaced. That rule existed to stop two live servers diverging; with one server there is nothing to diverge from. Four behaviours changed as a result, and they are written down as changed rather than quietly absorbed.
BYOK is a security decision before it is a cost one. A gateway that stores provider keys becomes the most valuable thing on the host. Forwarding the caller’s key costs a header per request and removes the category. The part that took the actual thought was the base-URL override: accepting an arbitrary upstream URL is an SSRF primitive, so it is constrained to the provider’s canonical host or an explicit allowlist, https only, and never a raw IP.
The loopback rewrite is the small detail that comes from having actually run it. Ollama advertises itself on 127.0.0.1, which inside a container means the container. Rewriting it to the host once, at the boundary, is the difference between a local model being a supported route and being a thing that only works when the process happens to be outside Docker.
- 7 hosted + 2 local
- Upstreams behind one API
- 0 lines
- Provider-specific code in the client