Knowledge Base

Open Build → Knowledge Base. A collection is a named group of documents; agents attach to collections, not to individual files.

Adding sources#

Add source covers one-shot ingestion — files, a web page, a repository. Connect links an external service that is synced on a schedule and kept deduplicated. Both land documents in the same collection and the same retrieval pipeline.

File upload#

Accepted extensions, exactly:

.txt   .md   .markdown   .csv   .tsv   .log
.html  .htm  .xml        .yaml  .yml  .json
.rtf   .pdf  .docx
.png   .jpg  .jpeg       .webp  .gif

Web page / crawl#

Give a URL and the page is fetched and converted to text. Use it for public documentation and policy pages. Pages behind a login cannot be fetched.

GitHub repository#

Ingests source and docs from a repository so an agent can answer questions about a codebase.

Public repositories work with no credentials at GitHub's anonymous limit of 60 requests an hour, which a large repository can exhaust mid-import. Set GITHUB_TOKEN on the deployment to raise that to 5,000 and to allow private repositories the token can read.

Pasted text#

The Manual tab takes a name and a body of text typed or pasted straight into the form — no file, no URL. It is the quickest way to add something that exists only in someone's head or in a chat thread: a support answer, an internal convention, a correction to a document you cannot edit. The text is chunked and embedded exactly like an uploaded file, so it is searchable on the same terms as everything else in the collection.

Connected services — Drive, Notion, SharePoint, Dropbox, Confluence, Website#

Connect opens a wizard for six providers. Credentials are pasted tokens (the platform's BYOK pattern — no OAuth consent screens to register), validated against the provider at save time, encrypted at rest, and never sent back to the browser: editing a source shows empty credential fields, and leaving them empty keeps what is stored.

Per source, listings cap at 500 items and each item's text at 400k characters. Anything the connector saw but did not ingest — an unsupported binary, a too-deep folder, the cap — is listed on the source card with its reason, never silently dropped.

Scheduled sync — indexing without duplicates#

Each connected source has a schedule: manual, hourly, daily or weekly. Scheduled syncs run on the platform's maintenance pass (the same engine that refreshes BI and SaaS data), and a claim on the source's next-run time guarantees that multiple app instances never sync the same source twice.

Change detection is two-level, and both levels exist to make an hourly schedule cheap:

1. 1

   Version skip — unchanged files are not downloaded

   The provider's change marker (modified time, revision, native hash) is stamped on each document. Unchanged marker ⇒ the item is skipped without being downloaded — a 400-file folder re-syncs for the price of a listing.
2. 2

   Content skip — unchanged text is not re-embedded

   Providers bump modified times on moves, permission edits and comments. Downloaded text is hashed (sha256); if it matches what is stored, the marker is refreshed and the document is not re-chunked or re-embedded. Embedding spend follows actual content change, nothing else.

Files deleted at the provider delete their documents (and chunks) here. The source card shows each sync's outcome as +added ~updated =unchanged −removed, and a database uniqueness constraint on (source, remote item) makes duplicate documents impossible even if everything above were wrong.

What happens on ingest#

file ──▶ extract text ──▶ split into chunks ──▶ embed each chunk ──▶ store
                             (few hundred words,      (vector = position
                              overlapping)             in meaning-space)

question ──▶ embed ──▶ nearest chunks ──▶ pasted into the prompt as [1] [2] …

Chunking matters more than people expect. Retrieval returns chunks, not documents — so if the sentence answering a question is split across two chunks, neither answers it well. Chunks overlap slightly to soften this.

Which model does the embedding#

Set this per collection under RAG settings → Embedding. Embeddings come from a connected model provider — the same place your chat models come from. There is no separate embeddings key and no dependency on an OpenAI account. OpenRouter is the suggested default, not a requirement: it works either through your own integration or, with no integration at all, through the operator's OPENROUTER_API_KEY, so a fresh install gets retrieval for free from the same account that already makes chat work. Any other connected provider exposing an OpenAI-compatible /embeddings endpoint can be selected instead.

The OpenRouter default is openai/text-embedding-3-small because it is the same 1536-d space the operator's OpenAI key produces: moving a collection onto OpenRouter to get off an exhausted OpenAI quota costs nothing and leaves existing chunks searchable. The other options are different spaces, so choosing one means re-indexing.

Every model in that list was called against OpenRouter's live endpoint and confirmed to return 1536 dimensions. That check is not ceremony: OpenRouter does not list embedding models in its public /models catalogue, so a plausible-looking id is no evidence the model exists. Two NVIDIA nemotron ids used to be offered here and both returned 404 No endpoints found — selecting one produced a failed embed with nothing to indicate the model had never been available.

Where the vectors are searched#

By default, in your own Postgres — kb_chunks.embedding is a pgvector column with an HNSW cosine index, and the permission check is the row-level security already protecting those rows. Nothing to connect, nothing extra to back up, and a collection that cannot half-exist because two systems disagree. For most deployments that is the end of it.

A self-hosted deployment can point the vector search at Qdrant instead, by setting VECTOR_STORE=qdrant and QDRANT_URL. The reason is capacity, not availability: an HNSW index wants RAM, and by default it wants it from the same instance serving your traces, audit, BI results and every other query. Past a few million chunks it is the largest thing in there, and the only way to feed it is to resize the whole database. Qdrant is a place to put the index that scales — and replicates — on its own. It does not make retrieval survive a Postgres outage: every hit is hydrated from the chunk rows in Postgres, so the database going down takes retrieval with it wherever the vectors live.

That setting is the default for every collection, not a decision for the whole deployment. RAG Settings → Retrieval → Vector index sets it per knowledge base, which is usually how to adopt Qdrant at all: the one collection that outgrew Postgres moves, every other one stays where its rows are. A collection pointed at Qdrant on a deployment that has none falls back to Postgres, and the picker says so rather than letting you find out from a server log.

One Qdrant node is not high availability. A single node is the right shape for a small install, and losing it degrades retrieval to keyword search rather than breaking it — but surviving the loss of a node means a Qdrant cluster with QDRANT_REPLICATION at 2 or more. Admin → Developer runtime → AI services shows which store is in use, whether it is answering, how many vectors it holds against how many chunks the database has, and the replication the collection actually got.

That same page has Re-index, which drops every vector in the store and writes them back from the chunks. It is the answer to every way an external index can drift — a restored-from-empty volume, a store switched on after documents were already embedded, a delete that happened while it was unreachable. It moves vectors that already exist; it does not re-embed, so a document that was never indexed stays keyword-only until it is.

Chunking modes#

Retrieval and generation want opposite things from a chunk. Matching is most precise when chunks are small and about one idea; answering is best when the model can see the whole passage. RAG Settings → Chunking → Chunking Mode decides how that tension is resolved for a document.

Parent-child sets two sizes: the parent is what reaches the model, and the existing chunk size becomes the child. A child is capped at half the parent, because a child the same size as its parent is flat chunking with extra bookkeeping. Parents do not overlap each other; children overlap within a parent.

Q&A exists because a question and a statement are different kinds of text, and that difference is a real part of the distance between their vectors. Asking “How do I rotate a key?” against a paragraph that says “Rotation issues a replacement…” is a harder match than asking it against the generated question “How do I rotate a key?”. It needs OPENROUTER_API_KEY; if generation fails, the run reports it rather than quietly writing flat chunks, so a collection never disagrees with the mode shown in its own settings.

Hybrid search and weighting#

Vector search finds meaning and blurs exact strings; an error code, a part number or a surname is exactly the kind of token embeddings smooth away. Keyword search is the opposite. RAG Settings → Retrieval sets which of them runs, per knowledge base. The same tab chooses the vector index that collection is searched in.

The weighting slider splits the score between them. Each retriever’s scores are normalised within its own list first, because cosine similarity (roughly 0.3–0.9) and text rank (roughly 0.0–0.3) are not comparable numbers — added raw, the slider would do almost nothing across most of its range. A chunk found by bothretrievers scores above one found by only one, which is usually the result you want.

Changing retrieval mode takes effect immediately and needs no re-embedding: it changes how the existing index is queried, not how it was built. Note that keyword search also indexes the generated question on Q&A rows, because the answer text often does not contain the words someone would search for.

Retrieval settings — the real numbers#

Reranking#

Configured per agent on the Knowledge tab: a Provider and a Re-rank model (for example llama-nemotron-rerank-vl-1b-v2).

• Cost — one extra model call per retrieval.
• When it pays — collections with many near-identical passages: long contracts, several revisions of one policy, product manuals for a family of similar products.
• When it doesn't — a small collection of clearly distinct documents. The first pass is already right.

Configuration by what you are indexing#

The settings interact, so choosing them one at a time tends to produce a collection that is individually reasonable and collectively wrong. These are five shapes of content that come up constantly, with the whole configuration for each and the reasoning behind it. Start from the closest one and adjust.

Policies, contracts, handbooks#

Long documents, formal language, and — the part that decides the configuration — several revisions of the same text, where the wrong version answering is worse than no answer.

Chunking

parent_child. A clause matches on its own wording, but a clause read without the section around it is how you get an answer that is technically present in the document and wrong in context.

Retrieval

Hybrid. Clause and section numbers ("7.2(b)"), defined terms and party names are exact strings, and embeddings smooth those away — hybrid keeps a keyword route to them.

Re-ranker

Yes. This is the case it exists for: five near-identical revisions all score well on similarity, and only a stronger model reliably puts the right one first.

top-K

5, or 8 if answers are being cut short. Bigger is not better — more chunks means more near-duplicates competing.

The prompt

Require the document name and effective date in the answer. Without that, a reader cannot tell which revision replied.

Product documentation and how-to guides#

Structured, headed, mostly distinct topics — the easiest case, and worth not over-tuning.

Chunking

flat. Headed sections are already the right size; parent_child mostly adds tokens here.

Retrieval

Hybrid, leaning semantic. Users ask in their words, but error codes and setting names must still hit exactly.

Re-ranker

Not at first. Distinct topics mean the first pass is usually right, and you would be paying a call per retrieval to reorder a list that was already correct.

top-K

5.

The prompt

Ask for numbered steps when the question is "how do I", and require a link or document name so the reader can continue in the source.

Support tickets and FAQs#

Short, repetitive, and written in answer language while your users ask in problem language — "it says invalid credentials" versus "authentication troubleshooting".

Chunking

qa. It embeds a generated question alongside the answer, which closes exactly that gap: keyword search also indexes the question, because the answer text often does not contain the words anyone would search for.

Retrieval

Hybrid. Error strings and product names are quoted verbatim by users.

Re-ranker

Yes once the collection passes a few hundred entries, where near-duplicate tickets start crowding each other out.

top-K

5–8. Several partial answers often combine into the right one here.

The prompt

Give it an escalation rule. A support agent that cannot answer should hand off, not improvise — see writing the system prompt.

A code repository#

Ingested from the GitHub source, so the collection is source files plus whatever docs live beside them.

Chunking

flat. Code has its own structure and splitting it by prose rules helps nobody.

Retrieval

Hybrid, weighted toward keyword. Identifiers are the query. Someone searching resolveEmbedTarget wants that symbol, not something semantically adjacent to it, and this is the one collection type where the keyword side usually deserves to win.

Re-ranker

Optional. Exact identifier matches are already decisive.

top-K

8. A question about behaviour usually spans a definition and its call sites.

The prompt

Require file paths in the answer, and say that quoted code must come from retrieved chunks rather than memory — a model's recollection of a well-known library is confident and frequently not your version.

One collection holding several kinds at once#

Usually how collections actually end up, rather than a choice anyone made.

Attaching a collection to an agent#

1. 1

   Create the collection and ingest into it

   Keep unrelated subject matter in separate collections — a mixed collection retrieves measurably worse.
2. 2

   Agent Builder → Knowledge → link it

   Linking auto-enables the kb_search tool.
3. 3

   Add the grounding instruction to the system prompt

   Without this the model happily falls back on general knowledge and you will not notice.
4. 4

   Turn on Citation Check

   Guardrails tab. It flags an answer that cites nothing when sources were available — see Guardrails.

Answer only from the provided sources. Cite them inline as [1], [2].
If the sources do not contain the answer, reply exactly:
"I don't have that in my documentation."
Never fill a gap with general knowledge.

Retrieval happens automatically before each turn, with numbered citations inserted. The kb_search tool additionally lets the agent search on demand, mid-answer, when its first read wasn't enough.

Graph search#

Ordinary retrieval finds chunks resembling the question. Graph search also follows relationships extracted between entities across documents, answering questions plain retrieval structurally cannot — "which suppliers are affected by the clause in appendix B?", where no single chunk contains both halves.

1. 1

   Build the graph

   Knowledge → Graph, on the collection. This is a separate, slower pass over the documents.
2. 2

   Enable the tool

   Agent Builder → Tools → kb_graph_search.

Sharing and access#

Collections are private to their owner. An administrator can grant a user or group read-only access from Access control; shared collections show a Shared badge and cannot be edited by the recipient.

Because the grant is enforced in the database, an agent's retrieval inherits it automatically — there is no second permission to keep in sync.

Per-source access scopes#

Connected sources add a second, finer layer: within a collection someone can already see, which of its synced documents may they retrieve? Each connected source picks one of three scopes, enforced at retrieval — vector and keyword paths alike, before any reranking model sees the text:

• Owner always retrieves their own documents, whatever the scope — a restriction you configured cannot lock you out of your own data.
• Fails toward deny. A provider that exposes no sharing info (Notion's API does not; a Dropbox plan without member listing) leaves documents owner-only, and the sync stats count them so the choice is visible. Tenant-wide links ("anyone in the organisation") match nobody but the owner — tenant membership can't be verified from here, and a wrong deny is recoverable where a wrong allow is not.
• Public embeds are anonymous. An embedded assistant retrieves only default-scope documents (and provider-public ones) — never "Only me" or ACL-restricted material, even though the embed runs under its owner's account.

When an answer is wrong#

Work through these in order. The cause is nearly always one of them.

Getting better results#

• Prefer structured documents. Headings give the splitter meaningful boundaries; an unbroken wall of text does not.
• Split by subject, not by department. One collection per coherent topic retrieves better than one collection of everything.
• Delete superseded versions. Three revisions of one policy retrieve interchangeably and the agent has no way to know which is current. This is the single most common cause of confidently outdated answers.
• Put numbers in tables. If the question is really arithmetic it belongs in Data Catalog — retrieval quotes prose, it does not compute.
• Name documents descriptively. The document name appears in the citation a reader sees, so refund-policy-2026.pdf beats final_v3.pdf.

Use cases#

Index your own documentation site#

1. 1

   Add Source → Website

   Give start URLs; optionally a sitemap URL, path prefixes to stay inside such as /docs, and a page cap (100 by default, 500 at most). No credential is needed.
2. 2

   Sync

   The crawler honours robots.txt, stays on the same site, prefers the sitemap and otherwise follows links breadth-first. A page version is the sitemap lastmod, else the ETag, else a content hash, so a scheduled sync re-fetches only what changed. A result like 5 documents indexed, 12 skipped by robots.txt is normal for a marketing site — narrow the prefixes to the documentation tree.

A Confluence space, code blocks intact#

1. 1

   Add Source → Confluence

   Site URL, the space keys to sync, an API token — plus the account email on Confluence Cloud. Cloud and Data Center are told apart from the URL and authenticated accordingly.
2. 2

   Pages arrive as text with their code macros preserved

   That is what makes a runbook useful to an on-call agent. Restrict retrieval by sharing the knowledge base read-only with the on-call group.

Share it, and delete it safely#

Sharing is read-only and done from Admin → IAM → Access. Deleting a knowledge base asks first and names what goes with it — every document, chunk and connected source, and the agents wired to it lose their knowledge. There is no undo, which is why there is a dialog.