Why does RAG avoid hallucinations?

An LLM makes things up when you ask it something it doesn't know. RAG hands it the real context (your fragments) at the moment of the question, so it answers from information you control instead of guessing. You can also ask it to cite the source of every statement.

What is an embedding?

A numeric vector that captures the meaning of a text fragment. Similar texts have nearby vectors, so searching for 'the fragments most relevant to a question' becomes a nearness search between vectors.

RAG or fine-tuning the model?

RAG for knowledge that changes (documents, policies, catalog): you update the index, not the model. Fine-tuning to teach a style or format, not facts. For 'answer with MY documents', RAG is almost always the right choice and cheaper.

How do I keep it from inventing outside the context?

In the prompt, instruct it to answer ONLY with the given fragments and to say 'I don't know' if it isn't there. And ask it for the citation: if it can't point to where the answer came from, that's a sign it made it up.

RAG explained: how to give an LLM your own information

A language model knows a great deal about the world, but knows nothing about your company: your manuals, your policies, your products. And if you ask it about something it doesn't know, it may invent an answer that sounds convincing. RAG (Retrieval-Augmented Generation) solves both problems.

The idea

Instead of expecting the model to "already know" your information, you give it to it at the moment of the question:

Index your documents: split them into fragments and turn them into embeddings (numeric vectors that capture the meaning).
When a question comes in, search for the fragments most similar to that question.
Pass the LLM the question together with those fragments and ask it to answer using them.

The model no longer guesses: it answers from a real context that you control. And you can ask it to cite where each fact came from.

The pattern in Python

# 1. Indexado (una vez): fragmentos -> embeddings -> base vectorial
#    (con librerías como sentence-transformers + FAISS, o un servicio gestionado)

# 2. En cada pregunta: recuperar los fragmentos relevantes
fragmentos = base_vectorial.buscar(pregunta, k=4)
contexto = "\n\n".join(fragmentos)

# 3. Generar la respuesta con el contexto
from anthropic import Anthropic
client = Anthropic()

resp = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=1024,
    system="Responde SOLO con la información del contexto. Si no está, dilo.",
    messages=[{
        "role": "user",
        "content": f"Contexto:\n{contexto}\n\nPregunta: {pregunta}",
    }],
)
print(resp.content[0].text)

Note the instruction in the system prompt: asking it to answer only with the context and to admit when it doesn't know is what drastically reduces hallucinations.

What is it good for?

Support assistants that answer with your real documentation.
Internal search in natural language over your manuals or wikis.
Onboarding: a chatbot that knows your processes.

Details that make the difference

How you split the documents (chunking) has a big effect on quality.
Good embeddings and a suitable vector database matter more than the model.
Citing the sources builds trust and allows verification.

RAG is today one of the most cost-effective ways to apply AI in a business: you retrain nothing, and the assistant improves just by updating your documents.

Want an assistant with your data?

At Xiliux I build RAG systems in Python from end to end. Tell me what information you want to put to work: contact.