Some requirements are not negotiable: data cannot leave the estate, no third party may see prompts, or the whole system must run in a network with no egress. Every layer of an agent has an option that satisfies this, though the trade-offs are real and worth being honest about.
Before designing for full self-hosting, establish what the requirement actually is. 'No data leaves our infrastructure' and 'no data is retained by a third party' are different constraints with very different costs. The second can often be met by a hosted provider under the right contract, or by a provider that does not retain prompts at all — which is a much cheaper path than running your own inference.
Be careful with the distinction: a provider that does not log your prompts is still a third party on the wire. That satisfies a retention requirement, not a residency one.
For genuine on-premise inference the choice is between simplicity and throughput. A local runner is the fastest path to something working on one machine. A dedicated serving stack is what you want once concurrency matters, and it is where the open-weight models become genuinely practical.
Open weights are also a hedge worth taking even when you use a hosted API: if the hosted option later becomes unacceptable, the same model runs on your own GPUs without a rewrite.
Memory and retrieval collapse into one dependency you probably already run. Web grounding, the layer teams assume forces a third party, has a self-hosted metasearch option. Observability, guardrails and containment are all available as software you deploy yourself.
Being honest about the trade: you take on GPU capacity planning, model upgrades, and the operational burden of every component in the stack. Open-weight models trail the frontier on the hardest reasoning tasks, though the gap on coding benchmarks has narrowed to roughly a point at the top of the leaderboards. Budget for the operational load, not just the licence savings.
Yes. Every layer has a self-hostable option: vLLM or Ollama for inference, Postgres with pgvector for memory and retrieval, SearXNG for web grounding, Langfuse for observability, Presidio and NeMo Guardrails for guardrails, and Kata Containers with an egress proxy for containment.
It depends which requirement you have. A provider that does not retain prompts satisfies a data-retention requirement but not a data-residency one — the data still crosses a third-party network. If your obligation is residency or air-gapping, you need inference on your own hardware.
Every page here answers to Accept: text/markdown and returns the same content at roughly a tenth the tokens. No separate site, no toggle — same URL.
curl -s -H "Accept: text/markdown" https://newagent.build/guides/self-hosted-agent-stack