2026-10-16 · Coldforge Agentic Services

How to Run Hermes Agent Locally with Ollama on an 8 GB Linux VPS (Zero Subscription Cost)

You can run a capable AI agent without a monthly model bill, even on modest hardware. This guide installs Hermes Agent, the open-source self-improving agent from Nous Research, and points it at a small local model served by Ollama on a Linux VPS with 8 GB of RAM. All inference stays on your machine.

The trade-off is real. Small models on a CPU are slower and weaker at tool calling than large cloud models. For chat, drafting, research and lighter agent tasks they work well and cost nothing ongoing.

What you need

• A Linux VPS with at least 8 GB of RAM and a few GB of free disk (10 to 15 GB is recommended if you pull two models)
• SSH access
• Basic command-line comfort
• No GPU required (performance will be limited by the CPU)

Step 1: Install and verify Ollama

Ollama is the simplest way to run open-weight models and expose an OpenAI-compatible API.

curl -fsSL https://ollama.com/install.sh | sh

Verify that the service is running:

ollama --version
curl http://127.0.0.1:11434/api/tags

You should see a JSON response. An empty model list, {"models":[]}, is fine at this stage. On most Linux installs the service starts automatically through systemd.

Step 2: Choose and pull one or two models

On 8 GB of RAM, stay with small models. Larger ones will swap heavily or fail to load cleanly. Sizes and context windows below come from the Ollama model library.

Pull the ones you want:

ollama pull qwen3:4b
# optional second model
ollama pull llama3.2:3b

Test a model quickly:

ollama run qwen3:4b
# type a short prompt, then /bye

Ollama loads models on demand and unloads idle ones. Two installed models are fine on disk, but on 8 GB of RAM you should only use one at a time.

Step 3: Install Hermes Agent

Use the official installer from Nous Research:

curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash

When it finishes, reload your shell so the hermes command is found:

source ~/.bashrc

Alternatively, Ollama can install and configure Hermes for you in one step:

ollama launch hermes

That command offers to install Hermes if it is missing, lets you pick a model, and points Hermes at your local Ollama. If you use it, skip to Step 5, and check afterwards that the context length matches what you set there.

Step 4: Point Hermes at your local Ollama server

Hermes needs to know where to send requests.

Option A: interactive setup

hermes setup
# or, to change the model later
hermes model

Choose Custom endpoint (under More providers), then enter:

• Base URL: http://127.0.0.1:11434/v1
• API key: leave blank (or type no-key)
• Model: the exact name Ollama shows, for example qwen3:4b
• Context length: 64000. Do not leave it on auto-detect, because Ollama reports the model's maximum window, not the one you actually configured.

Option B: edit the config file

Edit ~/.hermes/config.yaml:

model:
  default: "qwen3:4b"
  provider: "custom"
  base_url: "http://127.0.0.1:11434/v1"
  context_length: 64000

Step 5: Tune for 8 GB of RAM and a CPU

Raise Ollama's context window to 64,000 tokens

The context window cannot be set through the OpenAI-compatible API that Hermes uses, so set it on the server. Create a systemd override:

sudo systemctl edit ollama.service

Add these lines, then save and exit:

[Service]
Environment="OLLAMA_CONTEXT_LENGTH=64000"
Environment="OLLAMA_KEEP_ALIVE=24h"
Environment="OLLAMA_FLASH_ATTENTION=1"
Environment="OLLAMA_KV_CACHE_TYPE=q4_0"
Environment="OLLAMA_MAX_LOADED_MODELS=1"

Then reload and restart the service:

sudo systemctl daemon-reload
sudo systemctl restart ollama

What each line does:

• OLLAMA_CONTEXT_LENGTH sets the default window to 64,000 tokens.
• OLLAMA_KEEP_ALIVE keeps the model loaded for 24 hours, so the first request after idle is not delayed by a reload.
• OLLAMA_FLASH_ATTENTION reduces memory use as the context grows, and is required for the cache setting below.
• OLLAMA_KV_CACHE_TYPE compresses the context cache. The default, f16, is the most precise and the most hungry. q8_0 uses about half the memory and q4_0 about a quarter, with a small loss of precision that is more noticeable at large context sizes.
• OLLAMA_MAX_LOADED_MODELS stops a second model from loading beside the first.

Raise the Hermes timeout

Slow CPU responses should not fail. Create or edit ~/.hermes/.env:

HERMES_API_TIMEOUT=1800

Hermes already treats local endpoints generously, but setting this explicitly is harmless.

Step 6: Start Hermes and test

hermes

Begin with a simple chat. Then try a light tool-oriented task, such as asking it to list the files in the current directory or summarize a short text file. The first reply can be very slow, because Hermes sends its system prompt and tool list with every request and a CPU has to process them before it answers.

Optional: messaging gateway

Hermes can also expose itself through Telegram, Discord, Slack and similar platforms while still using your local model. To set it up, run:

hermes gateway setup

Then chat with the agent from your phone. The inference still happens on the VPS.

Troubleshooting tips

• Model fails to load or is extremely slow: free up RAM (htop or free -h), stop other services, and stick to a 3B to 4B model.
• Hermes rejects the model or complains about context: confirm context_length: 64000 in config.yaml and that the Ollama override from Step 5 is active. Use Hermes's built-in compression commands and keep sessions modest.
• Tool calls appear as plain text or fail: the model likely lacks strong tool support. Fall back to chat-style use or try a different model from the table.
• "No space left" during a pull: free disk space, or move the Ollama models directory with the OLLAMA_MODELS environment variable. On Linux the ollama user must be able to read and write the new directory.
• The first response is very slow: this is normal while the model loads and the prompt is processed. The keep-alive setting helps after that.

Final notes

This setup gives you a fully local, zero-subscription agent. It will not match a large cloud model on hard multi-step tool use, but for everyday questions, drafting, research and lighter automation it is capable and completely private.

Start with qwen3:4b, get a working chat session, then experiment with tools. You can add a second model later, or pair it with a free cloud tier for the occasional heavy task.

Sources

• Hermes Agent: run Hermes locally with Ollama
• Ollama: Hermes Agent integration
• Ollama FAQ and context length

Commands and model details were checked against these pages on October 16, 2026. Model sizes and tool support change often, so check them again before you build.