RAG, short for retrieval augmented generation, means making an artificial intelligence model look information up before it answers. The model no longer draws on its training memory. It draws on your documents. France's data protection authority, the CNIL, sums it up in one line. It is an information retrieval mechanism added on top of a vectorised database (generative AI Q&A, 18 July 2024).
The acronym comes from a research paper by Patrick Lewis and his co-authors (Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks). It was posted on 22 May 2020 and presented at NeurIPS the same year. Six years on, RAG has become the standard way to connect an AI to a company's own data.
The starting point is always the same. You ask a business question. The AI answers fast, phrased well, and off target. The problem is almost never the model. It is the missing context.
A model with no internal data does not know your processes. It has never seen your documents. It ignores your business rules and improvises as soon as the question gets specific. You get generic answers, sometimes right, often unusable. And sometimes answers that are wrong but convincing, which costs more than silence.
Tandem deploys this kind of system for mid-market companies. This guide covers what holds up in the field. The four ways to give an AI context, how RAG actually works, agentic RAG, and the three traps that sink a project in production. To frame the need before picking the technique, our AI audit method starts from use cases, not tools.
What exactly is RAG in AI
A RAG system always follows the same three-step logic. Much like a person who opens the right procedure before answering a customer.
- The question is asked by a user or by another system.
- The system retrieves the most relevant passages from your documents.
- The model writes its answer from those extracts alone, and can cite its source.
The key point fits in one sentence. The model does not answer from memory, it answers from your data. Inside a company, that changes everything as soon as business knowledge and precise rules are involved.
Four ways to give an AI context
Before building a RAG, look at the three simpler approaches. They cover a large share of real needs for a fraction of the work. RAG only comes fourth, and that ordering is deliberate.
Plain filters
This is the most underrated approach, and often the best place to start. The agent does not reason over an entire database. It applies precise filters, date, status, category, client. For the question « how many orders were confirmed yesterday for client X », it filters the relevant rows, computes and answers.
It is fast, cheap and unambiguous. The rule is simple. If a person would use filters in a spreadsheet, use filters.
SQL queries written by the AI
As soon as you need to aggregate, rank, average or compare periods, filters hit their limit. The agent then writes the SQL query that matches the database structure. For « what are our three most profitable products this quarter », it writes the query, hits the database, gets a structured result and explains it.
This is far more reliable than chaining filters and computing by hand. The rule still holds. If a person would build a pivot table, go through SQL.
Passing the whole document
Another classic approach hands the entire document to the model. Contracts, procedures, transcripts, reports. No technical complexity, full context, and very good results for a global analysis or a summary.
The limit is the context window. As of 19 August 2026, GPT-5.2 lists 400,000 tokens on its model card. Claude Opus 5 and Claude Sonnet 5 reach 1 million tokens on the API (Anthropic documentation). Other Claude models stay at 200,000 tokens.
A wide window does not settle the matter though. Anthropic's own documentation states it plainly, accuracy and recall degrade as the token count grows, a phenomenon it calls context rot. Filling the window is not a strategy. Curating what goes into it is one. Add the cost of processing a mass of useless text on every question, and the approach shows its limits quickly.
This approach stays the right one when the volume is known and the need is occasional. In every other case you need to be selective, and that is exactly where RAG earns its place.
Semantic search
Instead of filtering or sending everything, the system splits the documents, turns the text into vectors, then searches by proximity of meaning. Ask « how do we handle a customer dispute caused by a late delivery ». Even if no document contains that sentence, the system finds the right passages.
It is powerful because it works without exact word matching. It scales to thousands of documents. It is the right pick for documentation, procedures, customer reviews or meeting notes.
How a RAG works, from document to vector
Splitting into chunks
To search your documents, you have to prepare them first. That is the subtlety many teams discover too late. Documents are not stored whole. They are split into small pieces called chunks, turned into vectors by an embedding model, then stored in a vector database.
That version of RAG works, but it stays linear and passive. The system retrieves roughly relevant passages, sometimes misses the key one, and the model answers anyway. You get well-written answers that are occasionally wrong. In production, that is a real problem.
Going further means giving the system more control, more logic and genuine decision-making. That is what agentic RAG brings.
Agentic RAG, when the AI picks its own sources
Agentic RAG is no longer a simple pipeline. It is an autonomous agent driving every step. It does not just search then answer. It chooses which tool to use, checks whether the retrieved context is enough, runs another search when something is missing, and loops until it meets its goal.
I laid out this mechanism in a LinkedIn post on moving from RAG to agentic RAG in December 2025. The diagram below puts both pipelines side by side.
- Rephrase the query when the original question is too vague for a useful search.
- Select the sources, vectors, SQL, web, APIs or conversation memory.
- Iterate until it has enough context, within a fixed number of loops.
- Rerank the results to keep only the passages that genuinely matter.
- Generate a sourced answer, or run the loop again if it does not hold up.
Reranking
The principle is simple. Instead of retrieving three or four passages, the system retrieves ten, twenty, sometimes more. A second model then reorders the results by actual relevance, and only the best ones reach the model that writes the answer.
My advice comes down to two points. Put your effort into data pre-processing and into the reranking step. For the large majority of use cases that is enough, and it is what I was already writing on LinkedIn. Refining the agent comes afterwards, not before.
Metadata
When a document is split, context is lost. The source, the date, the category, the client, all of it disappears with the chunking. Metadata is what keeps it. It is added at vectorisation time, and the agent can then filter on it before running any semantic search.
Add at minimum the document date, its category and its identifier. That lets you search only the version of a procedure currently in force, or group the pieces of a single document.
The three traps of a RAG in production
A hand-built RAG works in two hours for a demo. It holds up far less well in production, and I said so bluntly on LinkedIn in January 2026. For learning, building it yourself is the right move. For production, three points are consistently underestimated.
- Document parsing. A badly split PDF returns incoherent answers. Basic extractors break the structure and flatten headings, tables and charts. On a financial or technical document, that is a deal breaker. Specialised components such as Docling handle this job properly.
- Synchronisation. A knowledge base is never frozen. Documents get deleted, moved, edited, and internal processes evolve. If the vector store is not aligned, the AI answers with stale content in full confidence. Duplicates are the most common trap. When a document changes, its old vectors must be deleted before new ones are created.
- Permissions and compliance. Without access control at vector level, any user can query any document. Data location, the processing agreement and data minimisation are not optional.
Automation is not about building everything yourself. It is about placing the cursor between control, risk and business value. Depending on the case, Tandem ships an off-the-shelf solution to move fast. Or delegates ingestion to a specialised component and keeps n8n on the business logic. Or goes self-hosted when sovereignty and long-term cost come first.
RAG or fine-tuning, which one to pick
The question comes up in every scoping session. The short answer fits in one sentence. Fine-tuning changes the model, RAG changes the context you hand it. They are not two competing options for the same problem.
| Criterion | RAG | Fine-tuning |
|---|---|---|
| What changes | The context sent on each call | The model itself |
| Data freshness | Immediate, the base updates | Frozen at training date |
| Traceability | The answer cites its sources | No source to show |
| Good for | Knowledge, procedures, documents | Tone, output format, jargon |
| Cost of a change | Reindex the affected documents | Run another training |
Start with RAG for almost every business use case. It exposes your data to a model without retraining it, keeps freshness and makes sources traceable. Fine-tuning becomes relevant when you need to durably change a model's tone, format or business vocabulary. The two combine well, but in that order.
A RAG agent at a hotel group
To move past theory, here is a real case. A hotel group runs several properties in France and receives thousands of customer reviews spread across Google, Booking and TripAdvisor. The teams face three problems. Too many reviews to read, recurring pain points that are poorly identified, and review responses handled ad hoc with no method.
The goal is an agent that analyses those reviews for management, spots friction points property by property and helps teams reply. Reviews are collected and stored in a SQL database. The internal response guidelines live in a vector database.
Vectorising reviews and procedures
A first workflow categorises each review automatically, then vectorises it and stores it in Supabase with its metadata, the property, the rating and the category. A second workflow watches a Google Drive folder. As soon as a document is added or edited, it triggers an update of the vector store.
That second workflow carries the detail I see forgotten most often. When an existing document is edited, its old vectors are deleted before the new ones are created. Without that step, the previous version of the procedure stays in the base and keeps showing up in answers.
The agent that queries the base
The agent searches relevant reviews semantically, groups weak signals and produces actionable summaries. It retrieves several passages with their metadata, then a Cohere reranker keeps only the top three. The model is called through OpenRouter, which leaves the provider choice open without touching the scenario.
Teams then ask questions in plain language. Summarise the negative reviews for the Annecy hotel. What are this month's main pain points in Toulouse. How should we reply to a four-star review mentioning breakfast. This kind of agent replicates across many other functions, starting with customer support and documentation roles.
The same mechanism feeds other building blocks. An AI voice agent that has to answer a procedural question relies on exactly this foundation. And if you want to reproduce the build yourself, our tutorial on creating an AI agent with n8n walks through the nodes one by one.
Should you put a RAG in place in your company
Yes, as soon as your answers depend on internal documents that keep moving. No, if a filter or a SQL query answers the question. That is the only rule that matters, and it can be tested in half a day.
RAG is not a hard technology. What is hard is keeping it alive in production. Parsing, synchronisation and permissions cost far more than the pipeline itself. A project that neglects those three ships an AI that answers wrong with confidence, which is worse than no AI at all.
This is also why keeping the system healthy matters as much as building it. Our AI autopilots cover that side, from tracking answer quality to keeping the knowledge base current.
Our recommendation has not changed in two years. Start small, on a bounded corpus and a precise use case. Get pre-processing and reranking right before refining the agent. Measure answer quality before opening the tool to the whole company. And plan the human side, because a good RAG nobody adopts is worth nothing, which our article on AI literacy covers in detail.
Bpifrance lists RAG among the strategic methods for French small and mid-sized companies in its December 2025 dossier. We share that view, with one nuance. The value does not come from the technique, it comes from the corpus you are willing to clean up.
