Title: LLM self-hosting: run the stack yourself %E2%80%94 LLM EU Canonical: https://llmeu.com/en/docs/self-hosting/ Markdown mirror: https://llmeu.com/en/docs/self-hosting.md Machine-readable: https://llmeu.com/llms.txt (index) - https://llmeu.com/llms-full.txt (full catalogue) - https://llmeu.com/llms-catalogue.json Get started - [Quickstart](/en/docs/quickstart/) - [Console](/en/docs/console/) - [Migration](/en/docs/migration/) - [API reference](/en/docs/api/) Reference - [Authentication](/en/docs/authentication/) - [Endpoints](/en/docs/endpoints/) - [Errors](/en/docs/errors/) - [Rate limits](/en/docs/rate-limits/) - [Streaming](/en/docs/streaming/) - [Policies](/en/docs/policies/) - [Routing and policy](/en/docs/routing/) - [Residency and regions](/en/docs/residency/) - [Evidence and verification](/en/docs/evidence/) - [OpenAI compatibility](/en/docs/openai-compat/) Operate - [Self-hosting](/en/docs/self-hosting/) # Self-hosting Node and a writable directory are the whole requirement. The statements below were read from the files named beside them; anything that could not be confirmed is not here. ## Entry points - package.json requires Node >= 22.5. The SQLite store is Node%E2%80%99s own node:sqlite module. - There is no dependency list at all: apps, packages, scripts and tests run on Node%E2%80%99s standard library, and the Dockerfile installs nothing. - Three processes exist; each opens the database and runs the migration at start-up. ``` node apps/web/server/main.mjs # hub + console + embedded API API_EMBED=false node apps/api/src/main.mjs # API only, no pages node apps/worker/src/run.mjs # probes, latency, rollup, housekeeping ``` ## Datastore DATABASE_URL=sqlite:./data/llmeu.db needs no server, and the directory and file are created on first open. The handle runs in WAL mode with foreign keys on and a 5 s busy timeout. packages/shared/src/db/schema.sql is the schema. It is create-only and idempotent, so re-running it is safe; a column added to an existing table must also be declared in the additive step in packages/shared/src/db/index.mjs. A postgres:// URL takes a separate adapter branch. That branch requires the pg package, which is not a declared dependency, and the header of schema.sql says to treat the Postgres path as unverified until a real driver and a tested migration exist. SQLite is the path this repository runs. ## Operator CLI node scripts/db.mjs reads DATABASE_URL and needs no services. Commands: ## Environment Grouped as .env.example groups them. ## Docker Compose and the image are optional: the repository%E2%80%99s own suite does not require Docker, and the primary path is a local process with no services. - The default profile runs web: ports 3000 and 8088, SQLite on the llmeu-data volume at /app/data, the embedded API, and a healthcheck on /healthz. - The same image runs the worker with node apps/worker/src/run.mjs, waiting for web to become healthy and sharing the llmeu-data volume. - Two extra profiles are off by default: postgres (postgres:16-alpine) and redis (redis:7-alpine, reserved and not read by the v1 code path). - The gpu profile runs vllm/vllm-openai with an NVIDIA device reservation and mounts weights read-only from the host; it is not enabled by default, because claiming EU inference without a GPU would be false. - The Dockerfile is FROM node:24-alpine with tini, no build step and no dependency install, runs as the node user, writes only under /app/data, exposes 3000 and 8088, and starts apps/web/server/main.mjs. ## Backups node scripts/backup.mjs with no flags writes ./backups/llmeu- .db. SQLite is copied with VACUUM INTO, never cp, because a live WAL database copied with cp can yield a torn or stale state. ``` node scripts/backup.mjs # ./backups/llmeu-.db node scripts/backup.mjs --out /srv/backups/ # explicit destination node scripts/backup.mjs --verify ./backups/llmeu-.db node scripts/backup.mjs --restore ./backups/llmeu-.db --yes node scripts/backup.mjs --list ``` Every SQLite backup is verified before the script reports success: it is restored into a temp copy, PRAGMA integrity_check has to return ok, the schema is re-applied, and row counts for models, organizations and users are printed. With a postgres:// URL the script uses pg_dump --format=custom and needs pg_dump on PATH; a dump cannot be opened as a database, so its row counts need a scratch restore. ## Mail Transactional mail %E2%80%94 password reset, address verification, invitations %E2%80%94 uses MAIL_SMTP_* or MAIL_API_URL with MAIL_FROM. The transport is chosen once at start-up and reported on every page that needs mail. A deployment with neither is a supported state: sign-up, sign-in, password change, teams and invites work, but password-reset and verification mail cannot be processed there. The pages say so, and node scripts/db.mjs passwd is the recovery route. ## Inference backends An endpoint row names a runtime and a region, and the backend registry picks the adapter by region, so an endpoint can only be answered by a machine in the region it names. - INFERENCE_MOCK=true keeps the deterministic development backend, which needs no GPU and no weights. - GPU_ENABLED=false keeps managed backends out; with GPU_ENABLED=true, VLLM_BASE_URL applies to the single region named by VLLM_REGION. - A deployment with several regions sets one variable per region, named after the region slug (for example VLLM_BASE_URL_DE_FRA). An endpoint in a region with no configured backend is refused, rather than served from a machine in another region. - Partner backends come from PARTNER_BASE_URL_ and take PARTNER_SOVEREIGNTY and PARTNER_API_KEY; the server allows partner use when PARTNER_BASE_URL is set. With no GPU configured, the deterministic development backend answers and its output says no real weights ran and that first-party inference needs VLLM_BASE_URL and the gpu_enabled flag. A stand-in success that looked real would be the one thing this product must never ship.