Supported file types
Ragen AI accepts a wide range of formats. When you upload a file, the background worker automatically parses it into searchable chunks.Document processing pipeline
When you upload a file, Ragen AI processes it automatically in the background. You do not need to trigger any additional steps — the document becomes searchable once processing completes.1
Upload
Your file is transferred to secure cloud storage. The upload returns immediately; processing happens asynchronously.
2
Parse and chunk
A background worker parses the file using a format-specific splitter and divides the content into overlapping chunks sized for retrieval.
3
Generate summary
The worker generates a short summary of the entire document and prepends it as a special summary chunk. This gives retrieval a high-level anchor that no individual flat chunk contains. Summaries can be turned off for the whole deployment (
FEATURE_FLAG_DOC_SUMMARIES=0) or per organization in the admin panel; documents indexed earlier keep theirs.4
Embed
Each chunk — including the summary chunk — is converted to a hybrid representation: a dense vector using
bge-multilingual-gemma2 for semantic meaning, plus BM25 sparse tokens for exact-term matching.Before the dense embedding, the worker adds a short prefix to each chunk: the document’s title, the heading the chunk sits under, and the first sentence of the document’s summary. A passage that only says “the notice period is 30 days” then answers a question that names the contract. Building the prefix costs no model call. The worker stores it beside the chunk, never inside it, so the answer model and the citation still show you the text exactly as written. BM25 indexes the bare chunk.5
Store
Both vector types are stored in the vector collection along with the file’s visibility rules as chunk metadata. Retrieval queries automatically filter results based on who is asking, so a document someone cannot read never surfaces in their answers.
Documents indexed before a pipeline change
The title-and-section prefix is on by default (contextualChunks), and the worker adds it as it indexes a document. Documents you uploaded before it was on keep working without the prefix until you re-index them. If you are an organization admin, open Organization → RAG pipeline: it shows how many documents carry the prefix, and you can re-index the rest in the background. Re-indexing re-embeds the version of each document you see, including your edits.
Folders
Folders let you group related documents and apply permissions at a hierarchy level rather than file by file. Ragen AI supports arbitrarily deep nesting — a folder can contain both files and sub-folders. Each folder can be:- Personal — owned by you, visible only to you and org admins unless explicitly shared
- Team folder — assigned to a team, visible to all team members and org admins automatically
- Shared — explicitly shared with specific users or teams at either
vieworfullaccess level
File visibility rules
Every file’s visibility is determined by who owns it and whether it has been explicitly shared. The table below covers the common scenarios.Visibility rules are enforced at retrieval time inside the vector store — not just in the UI. A document someone cannot read will never surface in their answers, even through the API.
Document versioning
Every time a document’s content changes, Ragen AI creates a new version and re-indexes it. This gives you a full audit trail and the ability to roll back to any previous state.
Rolling back appends a new version with the earlier content rather than deleting the intermediate versions, so you can always undo a rollback.
RAG readiness score
A file can show aRAG NN/100 badge in the file list and grid. It is a score out of 100 that one LLM call gives the document. By default, a document is scored when you choose Analyse in its Optimize tab and it has no score yet. That score is the baseline its suggestions are measured against, and it is kept until the text changes. Uploading or re-processing a file does not score it unless an administrator has turned on Score every upload (below), so a new file shows no badge until someone analyses it. The same number appears in Version history and as the current score in the Optimize tab.
The rubric rewards question-and-answer prose: numbered headings, sections of 80–150 words, and sections that stand on their own. It reads the first 12,000 characters of the indexed text, about three pages.
Keep these limits in mind when you read it:
- Tables score low even when they are indexed well. A spreadsheet or price list can barely earn three of the five criteria. Re-indexing spreadsheets into proper table chunks lowered their scores, even though the same kind of change measurably improved retrieval.
- Long documents are judged on their opening pages.
- Scores are not comparable across document types. Don’t rank a price list against a policy document.
- Score every upload makes each upload and re-process score the document automatically, as it did before scoring became on request. It is off by default, because it costs one call per file and the score predicts how well a single document will be retrieved only weakly. It is most useful beside an edit, as the before and after in Optimize.
- RAG readiness score turns scoring off altogether. With it off, nothing is scored, even with Score every upload on: the badge disappears, and Optimize still suggests improvements without showing a score.