# The self-hosted agent stack _Updated 2026-08-21._ Some requirements are not negotiable: data cannot leave the estate, no third party may see prompts, or the whole system must run in a network with no egress. Every layer of an agent has an option that satisfies this, though the trade-offs are real and worth being honest about. ## The constraint is usually narrower than it sounds Before designing for full self-hosting, establish what the requirement actually is. 'No data leaves our infrastructure' and 'no data is retained by a third party' are different constraints with very different costs. The second can often be met by a hosted provider under the right contract, or by a provider that does not retain prompts at all — which is a much cheaper path than running your own inference. Be careful with the distinction: a provider that does not log your prompts is still a third party on the wire. That satisfies a retention requirement, not a residency one. In the registry: [Venice AI](https://newagent.build/c/venice), [Amazon Bedrock](https://newagent.build/c/bedrock), [Google Vertex AI](https://newagent.build/c/vertex) ## Inference on your own hardware For genuine on-premise inference the choice is between simplicity and throughput. A local runner is the fastest path to something working on one machine. A dedicated serving stack is what you want once concurrency matters, and it is where the open-weight models become genuinely practical. Open weights are also a hedge worth taking even when you use a hosted API: if the hosted option later becomes unacceptable, the same model runs on your own GPUs without a rewrite. In the registry: [Ollama](https://newagent.build/c/ollama), [vLLM](https://newagent.build/c/vllm), [LiteLLM](https://newagent.build/c/litellm), [DeepSeek](https://newagent.build/c/deepseek) ## Everything else has an open-source answer Memory and retrieval collapse into one dependency you probably already run. Web grounding, the layer teams assume forces a third party, has a self-hosted metasearch option. Observability, guardrails and containment are all available as software you deploy yourself. In the registry: [pgvector](https://newagent.build/c/pgvector), [SearXNG](https://newagent.build/c/searxng), [Langfuse](https://newagent.build/c/langfuse), [Microsoft Presidio](https://newagent.build/c/presidio), [NeMo Guardrails](https://newagent.build/c/nemo-guardrails), [Kata Containers](https://newagent.build/c/kata), [iron-proxy](https://newagent.build/c/iron-proxy) ## What self-hosting costs you Being honest about the trade: you take on GPU capacity planning, model upgrades, and the operational burden of every component in the stack. Open-weight models trail the frontier on the hardest reasoning tasks, though the gap on coding benchmarks has narrowed to roughly a point at the top of the leaderboards. Budget for the operational load, not just the licence savings. ## Common questions ### Can an AI agent run entirely on-premise? Yes. Every layer has a self-hostable option: vLLM or Ollama for inference, Postgres with pgvector for memory and retrieval, SearXNG for web grounding, Langfuse for observability, Presidio and NeMo Guardrails for guardrails, and Kata Containers with an egress proxy for containment. ### Is a no-logging hosted provider enough for compliance? It depends which requirement you have. A provider that does not retain prompts satisfies a data-retention requirement but not a data-residency one — the data still crosses a third-party network. If your obligation is residency or air-gapping, you need inference on your own hardware. ## Machine interfaces - `GET https://newagent.build/api/registry` — every component as JSON - `GET https://newagent.build/api/vendors` — vendors, with the repo and releases feed to watch - `GET https://newagent.build/api/advise?q=` — recommended stack for a description - `GET https://newagent.build/api/stack?=&format=sh|json|md|yml|agents|env` — a stack as files - `GET https://newagent.build/llms.txt` — the whole registry in one fetch Any page here also returns markdown if you send `Accept: text/markdown`.