How to Self-Host an AI Assistant with Hermes

I run Hermes Agent, the open-source assistant from Nous Research, on an Ubuntu server hosted by Oracle Cloud and talk to it through Telegram. Tailscale provides private administrative access, while a hosted model does the inference. That split matters: I self-host the assistant runtime, not the model.

Decide what you are hosting

An assistant has several parts: the gateway that receives messages, stored conversations and memory, tools that can touch files or services, and the language model. Installing the first three on your server does not automatically self-host the model.

If you configure OpenAI, Anthropic, OpenRouter or another hosted provider, prompts and the relevant conversation or tool results go to that hosted model provider. Search, messaging and other integrations also send data to their own services. Local storage gives you control over the agent's state; it does not keep all data on your server.

Running inference locally is a separate decision. Hermes supports local models, including managed llama.cpp and compatible local endpoints. Local inference needs suitable hardware and may produce weaker or slower results than a hosted frontier model. Pick it because the privacy, offline use or control is worth that trade-off, not because "self-hosted" sounds purer.

A small Hermes setup

Start with a Linux machine you can maintain, Hermes Agent, one chat interface and one model route. A spare computer, dedicated VM or modest VPS can run the gateway when inference happens elsewhere.

Upgrading Oracle Cloud to Pay As You Go (PAYG) does not remove its free Ampere A1 compute allowance. Oracle's current price list gives each paid tenancy 3,000 OCPU-hours and 18,000 GB-hours of memory per month, enough for 4 OCPUs and 24 GB of RAM running continuously. This is a shared allowance, not a separate allocation per instance. Usage above it, or other resources outside their own free allowances, can still be billed.

I use Hermes Agent. Its official installation guide supports Linux on x86-64 and ARM64. Follow that page rather than copying an old command from a blog; installation details change. Docker is optional. It can provide useful containment and repeatable deployment, but the supported native installer is simpler for many machines. A separate VM or dedicated host can provide a stronger boundary than a loosely configured container.

For chat, I use Telegram. Create a bot with BotFather, then follow the current Hermes Telegram setup. Restrict the bot to your own Telegram user ID or use pairing; an unprotected bot is an exposed control surface. By default, Hermes fetches messages through outbound requests to Telegram. Tailscale is not required for messaging.

I use Tailscale for private administrative access. Tailscale Serve can make a dashboard available within your tailnet. Keep the underlying service on localhost, restrict tailnet access, and do not substitute Funnel: Funnel publishes a service to the internet.

Model access and cost

Choose one provider route first. API keys are the most portable option and are usually metered. Hermes also supports provider-specific OAuth or sign-in routes, including an OpenAI Codex route for eligible ChatGPT or Codex subscriptions. This is not generic access to every OpenAI model, and eligibility, limits and terms can change. The provider matrix is the useful source of truth.

Keep the billing categories separate: ChatGPT subscription access applies only through supported, eligible routes; ordinary OpenAI API usage has separate billing. The same caution applies elsewhere. A subscription login, an API account and a local model are different products, even when one assistant can switch between them.

A budget should include the server, model calls and paid tools. Set provider limits or alerts where available, then review actual usage. Avoid building the architecture around a promotional free tier or an exact price that may disappear next month.

How Hermes remembers and reuses what it learns

Hermes separates persistent memory from reusable skills. Its built-in memory keeps compact notes in MEMORY.md and a user profile in USER.md. These carry useful facts and preferences into new sessions without loading the entire conversation history.

Skills are instructions for particular tasks, loaded when needed. Hermes can save a procedure it has worked out and reuse or improve it later. For example, a skill could record how to build and test a website, rather than making the assistant rediscover those steps each time. That is more useful than filling permanent memory with every command from a finished job.

Keep memory and skills reviewable, backed up and easy to correct. They can contain mistakes or outdated instructions. Check what has actually been saved, and treat instructions from unfamiliar sources with the same caution as other untrusted input.

Security is the main design job

Run the assistant as a dedicated non-root account and apply least privilege to reads as well as writes. Read access is not harmless: an agent that can read SSH keys, browser profiles, client files or environment secrets can disclose sensitive data through a model call or network tool. Mount or expose only the directories and credentials needed for the task.

Treat email, web pages, documents and repository instructions as untrusted input. Prompt injection can persuade an agent to reveal data or misuse a tool. Filters help, but they are not a security boundary. Use an isolated execution environment, narrow filesystem mounts, limited credentials and restricted network access. Hermes documents these limits in its security guide.

Require human approval for consequential actions: sending messages, publishing, deleting data, spending money, changing credentials or touching production. Review the exact target and payload, not just a vague action label. For low-risk automation, start read-only with a narrow scope and expand only after observing it.

Keep secrets outside version control, update the host and agent, back up the state you care about, and test recovery. Self-hosting trades product limits for operational responsibility. It is worthwhile when you want a persistent, adaptable assistant and are prepared to maintain the boundary around it.