RAG

The pipeline has two halves. In the preparation phase documents such as policies, manuals, service descriptions and support history are split into chunks, each chunk is converted into an embedding and stored in a vector store. In the answering phase the user question is embedded as well, the closest chunks are retrieved, often re-ranked by a more precise model, and inserted into the prompt with an instruction along the lines of "answer only from these materials and say so when they do not cover it". The model then writes the answer and cites the documents it used.

The comparison with fine-tuning explains why RAG became the standard answer to "a bot over our documents". Fine-tuning transfers style and format well but facts poorly, costs more, needs data in a specific shape and goes stale with the documents: update a policy and you retrain. RAG updates by reindexing, answers from the current version, lets a human verify through the citation, and supports access control so an employee only sees what they are entitled to. For an internal knowledge base, support desk or catalogue assistant those properties are decisive.

RAG quality is decided by retrieval rather than by the model, and that is the main practical discovery for most teams. If the right chunk is not found, the strongest model will answer wrongly or decline. So the points that matter are chunking (too small loses context, too large fills the prompt with noise), whether results are re-ranked, whether synonyms and industry jargon are handled, and whether vector search is combined with ordinary full-text search. And one mandatory element: a set of test questions with reference answers, run against every change to the pipeline.

Related terms

• LLMAn LLM is a large language model: a neural network trained on vast amounts of text to predict the next token, which is why it can write, summarise and answer, yet it stores no facts the way a database does and can be confidently wrong.
• EmbeddingsEmbeddings are numeric vectors a model produces from text, images or products so that items close in meaning end up close in the vector space, which is what makes semantic search, recommendations and duplicate detection possible.
• Prompt engineeringPrompt engineering is the design of instructions for a language model: role, context, task, output format and constraints are written so the result is reproducible, and quality is judged against a set of test cases rather than by impression.
• AI agentAn AI agent is a program in which a language model does not merely answer but acts: it receives a goal, chooses its own tools such as search, APIs, a database or sending mail, works through steps in a loop and stops when the task is done or a limit is hit.
• Russian personal data law (152-FZ)152-FZ is the Russian personal data law: it requires that data on Russian citizens be collected in databases located in Russia, that the regulator be notified of processing, that consent be obtained separately, and that any breach be reported within 24 hours.

Related services

• AI and ML developmentWe build AI that solves a defined task and pays for itself, not demos for the sake of demos. Classifiers, recommendations, text and document processing, LLM assistants, and model integration into an existing product.
• IT consulting and product auditThe most expensive mistakes in software happen before the first line of code: a misread problem, a stack chosen out of a contractor’s habit, and a specification that does not exist. Consulting exists to settle all of that before the development meter starts running: what to build, out of what, at what cost and in which order. The result is a document, not an opinion on a call.
• Web app and Telegram Mini App developmentFor when a site is no longer enough and you need a product: an account area, a dashboard, an internal tool or a Mini App inside Telegram. We design the architecture, write the backend and frontend, and take it to release.

Read more

• An AI Assistant on Your Own Knowledge Base: How RAG Works and What It CostsHow a RAG assistant differs from a button-tree bot and from an LLM with no data, where hallucinations come from and what actually reduces them, what SaaS costs against a custom build, and which month the project breaks even.
• Where AI and ML Actually Pay Off in a Business, and Where They Are Expensive HypeMachine learning pays off where there is a repeated decision, a lot of similar data and tolerance for error. Missing any one of the three means do not. Here are the use cases that work, their realistic accuracy, the running costs and how to run a four to six week pilot.