What is a context window, and what is a token?
A token is a small piece of text, roughly a word fragment. A context window is the maximum number of tokens one call may hold: instructions, the conversation so far, retrieved passages and the reply.
LLM EU
Frequently asked questions
Basics
LLM vs AI: what is a large language model, what is an llm base, and what is Nemotron 3 Super?
AI is the field; a large language model is one kind of model inside it, a base LLM continues text instead of following instructions, and Nemotron 3 Super is a 123.6B mixture of experts.
What is an LLM?
An LLM is a program trained on large amounts of text to predict the next token, a small piece of text. It answers from its weights and the prompt you send, not from a database it searches.
Retrieval
What is an LLM retriever, and which multilingual embedding model should I use?
A retriever pairs an embedding model that turns text into vectors with a reranker that scores passages. multilingual-e5-large is MIT with 512-token chunks; embeddinggemma-300m takes 2048.
What is RAG?
RAG is a pipeline, not a model: your application retrieves passages and the model answers from those. Only the last stage is a chat completion; embedding and reranking are separate calls.
Hosting
How do LLMs run in the EU?
A model runs where an endpoint runs, and the row records its region and class. Residency is a filter applied before scoring: if nothing satisfies it, the request returns 409 no_endpoint_for_policy.
Open weights or an API: which should I use?
Open weights mean the files are downloadable; an endpoint means someone runs them for you. Many European deployments use both: an API for variable load, a self-hosted checkpoint for sensitive work.
Cost
What does an LLM API call cost, and what do the benchmark numbers mean?
A price is read from the rate an endpoint row records, per million tokens in and out, plus one visible fee per request. A row with no recorded rate is shown without a price.
Licences
How do open model licences work?
A licence is a recorded field on every model card: an SPDX identifier and a link to the text. Open weights means the weights are downloadable, not that commercial use is allowed.
API
What are LLM API rate limits?
A rate limit here is a ceiling per bucket in a fixed window: 600 calls per key and 1800 per organization each minute, with at most 8 requests in flight on one key.
Routing
How does an LLM router work?
A router turns a request naming no machine into one endpoint response. A pure function filters candidates by policy and residency, scores what survives and takes the highest score.