Goethe user guide

What you can use it for

• Easy to serve. Every model you load is served at the same URL; a request picks its model by name.
• Fast. Use the custom kernel attachments to accelerate and optimise model capability.
• A private API for your team. One key per person or app; nothing reaches the API until you allow it.
• Home or office over a VPN. Serve from a GPU box and use it from your laptop through Tailscale, WireGuard or ZeroTier.
• Your own apps and tools. Point OpenAI, Anthropic or LiteLLM clients, Open WebUI, coding assistants and similar tools at hardware you control.
• Many models, many GPUs. Compare models by name on the same address, split large models across GPUs, or run on cloud GPUs when you need them.

Quick install

curl -fsSL https://meshapplied.com/install.sh | bash
goethe start

goethe start starts Goethe and walks you through three steps: install a model, serve it through the API, and configure who can reach it. If you prefer plain commands:

# 1. Start Goethe and load a model
goethe up
goethe install --model zai-org/GLM-5.3-Flash --load

# 2. Share it
goethe configure api                 # this machine only, your VPN (recommended), or a public address
goethe key --create my-laptop        # one key per app or person
goethe api                           # the address to give your apps

goethe status tells you whether Goethe is running, with the API address and your keys.

→ Hardware requirements
→ Software requirements

API compatibility

The API is at https://<address>:<port>/v1. It speaks three schemas at once, so OpenAI, Anthropic and LiteLLM clients all work against the same address and key:

• OpenAI: chat, completions, responses, embeddings, rerank, models.
• Anthropic: POST /v1/messages (and token count); errors on this route use Anthropic's format.
• LiteLLM proxy: the same routes without /v1, plus /model/info, /model_group/info, liveliness and readiness.

Send the key as Authorization: Bearer … (OpenAI), x-api-key (Anthropic), x-litellm-api-key (LiteLLM) or api-key (Azure-style). The API uses HTTPS with a certificate Goethe creates; clients skip verification (curl -k, verify=False) or trust <goethe-home>/nginx/certs/server.crt.

curl

curl -k https://<address>:8443/v1/chat/completions \
  -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"model":"zai-org/GLM-5.3-Flash","messages":[{"role":"user","content":"Hello!"}],"max_tokens":128}'

OpenAI SDK

import httpx
from openai import OpenAI

client = OpenAI(
    base_url="https://<address>:8443/v1",
    api_key="goethe_sk_...",
    http_client=httpx.Client(verify=False),
)
reply = client.chat.completions.create(
    model="zai-org/GLM-5.3-Flash",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(reply.choices[0].message.content)

Anthropic SDK

import httpx
from anthropic import Anthropic

client = Anthropic(
    base_url="https://<address>:8443",
    api_key="goethe_sk_...",
    http_client=httpx.Client(verify=False),
)
reply = client.messages.create(
    model="zai-org/GLM-5.3-Flash",
    max_tokens=256,
    messages=[{"role": "user", "content": "Hello!"}],
)
print(reply.content[0].text)

LiteLLM

import litellm

reply = litellm.completion(
    model="litellm_proxy/zai-org/GLM-5.3-Flash",
    api_base="https://<address>:8443",
    api_key="goethe_sk_...",
    messages=[{"role": "user", "content": "Hello!"}],
    ssl_verify=False,
)

In other tools that take a custom OpenAI endpoint: set the base URL to https://<address>:<port>/v1, your Goethe key, and a model name from goethe api.

Limits: about 20 requests per second per client address; per-key --rpm; bodies up to 10 MB. Unloading a model finishes in-flight requests first; new ones get 503 with Retry-After.

CLI

Every value goes after its flag: goethe unload --model …, goethe gpu --allow gpu0. Listing commands accept --json. Exit codes: 0 success, 1 error, 2 wrong usage, 3 needed confirmation without a terminal (add --yes).

Everyday

goethe up          # start; also starts at boot from now on
goethe status      # running or not, API address and keys
goethe down        # stop; won't start at boot until the next up
goethe ps          # what is running
goethe api         # address, keys and loaded models
goethe logs --model zai-org/GLM-5.3-Flash -f
goethe logs --daemon
goethe doctor
goethe hw profile

Start and stop

Models

Running

Sharing

GPUs and system

Add models

goethe recommend
goethe install --model zai-org/GLM-5.3-Flash
goethe install --model zai-org/GLM-5.3-Flash --load

The model id is the Hugging Face repo name. Useful flags: --load, --ctx, --max-conc, --trust-remote-code (only for models you trust), --hf-token for gated or private models (asked once per download, never stored).

For a gated or private model: accept the license on Hugging Face, create a read token, then goethe install --model <org>/<model> --hf-token. In goethe start, choose “A private or gated model”.

To register a folder you already have, put it under Goethe's models directory and run goethe scan, or:

goethe install --model /data/models/my-finetune --name me/my-finetune

goethe models
goethe remove --model zai-org/GLM-5.3-Flash

Load and run

goethe load --model zai-org/GLM-5.3-Flash

Goethe picks GPUs, sizes memory and waits until the model answers. The first load on a new machine can take a few extra minutes; later loads are faster. Examples:

goethe load --model zai-org/GLM-5.3-Flash --gpu gpu1
goethe load --model zai-org/GLM-5.3-Flash --tp 2
goethe load --model zai-org/GLM-5.3-Flash --gpu gpu0 gpu1
goethe load --model zai-org/GLM-5.3-Flash --ctx 65536 --max-conc 32
goethe load --model zai-org/GLM-5.3-Flash --replica
goethe config --model zai-org/GLM-5.3-Flash --ctx 65536 --max-conc 32

Load as many models as your GPUs can hold; they share one API address. Use goethe ps to see what is running.

goethe unload --model zai-org/GLM-5.3-Flash
goethe unload --all
goethe unload --model zai-org/GLM-5.3-Flash --force

If a GPU is already used by other programs, Goethe asks before loading beside them; it never stops processes it did not start. Use --yes in scripts, or goethe gpu --deny gpu0 / --allow to control which GPUs Goethe may use.

Share the API

goethe configure api

• Local only: programs on this machine.
• VPN (recommended): Tailscale, WireGuard or ZeroTier.
• Public network: any reachable address; you open the port yourself.

goethe key --create my-laptop
goethe key --create ci --rpm 600 --expires 30d
goethe key --create chat-only --models zai-org/GLM-5.3-Flash
goethe key
goethe key --revoke <key-id>

Every request needs a valid key (401 otherwise). Options on create: --expires, --rpm, --models, --admin, --raw.

goethe allowlist --add 192.168.1.0/24 --description "office LAN"
goethe allowlist --add 203.0.113.7
goethe allowlist --remove 192.168.1.0/24
goethe api
goethe alias --set chat --model zai-org/GLM-5.3-Flash
goethe domain
goethe domain --check api.example.com

For a custom hostname, add a DNS A record to the address goethe domain shows; clients use https://api.example.com:8443/v1 with the same certificate settings as for the IP.

Hardware requirements

Older GPUs (Volta, Turing and earlier) are not supported. A machine can mix generations; each model runs on GPUs of one generation. Large models can be split with --tp (and --ep for MoE). 4-bit NVFP4 models run on every supported GPU.

Goethe ships its own CUDA 13 runtime. You only need the NVIDIA driver:

GeForce and TITAN cards need driver 580 or newer. Check with nvidia-smi. Enough GPU memory for the model is the other hard requirement; goethe recommend lists what fits.

Software requirements

• Linux on x86_64 or ARM64 with glibc 2.30 or newer (for example Ubuntu 20.04+ or Debian 11+).
• Disk space for your models, plus about 12 GB for Goethe itself.
• Goethe brings its own CUDA runtime and compiler; you do not install a CUDA toolkit. The installer checks the machine and can install missing system pieces after asking. Run goethe doctor any time for a check with fixes.

After install, open a new terminal if goethe is not found yet. Files live under /workspace/.goethe on cloud volumes, /var/lib/goethe as root, or ~/.goethe otherwise.

goethe update --check      # is there a new version?
goethe update              # install it; loaded models keep serving
goethe uninstall           # remove Goethe; add --keep-models to keep models

Where it runs

Goethe recognises the cloud provider or container, keeps files on a persistent volume when there is one, and shows the address to use from outside. In a container, the GPU and driver come from the host.

Supported models

Hugging Face format (config.json plus .safetensors). Architectures include Llama, Qwen, DeepSeek, Mistral, Gemma, Phi, GLM (including GLM-5.3 Flash), GPT-OSS, Kimi, MiniMax, Nemotron, Granite, Command R, OLMo, InternLM, InternVL, embeddings and more. Dense, MoE, hybrid and vision-language models are supported. goethe install / goethe scan say immediately if an architecture is not supported.

Format is detected from the files. GGUF and .bin pickle weights are not supported; use safetensors.

Settings

Optional. Set in your shell or in goethe.env inside Goethe's folder so the background service picks them up.

Troubleshooting

goethe: command not found: open a new terminal, or export PATH="$HOME/.local/bin:$PATH".

goethed is not running: run goethe up. Check with goethe status.

Unknown model: use an id from goethe models.

A load fails: Goethe prints the reason and the end of the log; full log: goethe logs --model … -n 200. Common fixes: free GPU memory (--ctx, --max-conc, --tp, or a smaller / quantized model); --trust-remote-code when the model needs it; goethe doctor for driver and GPU issues.

Anything else: goethe doctor, then goethe logs --daemon.