RAG Internal Knowledge Base for Enterprises: 2026 Cost and Architecture

The verdict in three sentences

An internal knowledge base with AI (RAG, retrieval-augmented generation) lets employees ask a question in plain language and get a sourced answer linking to the original document, instead of digging through tens of thousands of files. For a mid-size enterprise, a custom RAG project costs 25,000 to 80,000 EUR excl. tax (about 27,000 to 87,000 USD) and 200 to 2,000 EUR per month of inference, with 2 to 4 months to reach over 85 % correct answers on a business test set. Microsoft 365 Copilot remains relevant for office productivity, but it does not replace a controlled RAG when your procedures are messy, multi-source or subject to fine-grained access rights.

Copilot, off-the-shelf solution or custom RAG

An IT department has three options in 2026. Microsoft 365 Copilot relies on your tenant index and respects SharePoint permissions, but controls neither document chunking nor source quality. Off-the-shelf solutions (enterprise search engines with AI) plug into several sources. A custom RAG gives you control over every building block.

Custom RAG costs more upfront, but its per-user cost collapses beyond a few hundred users, because you pay for inference rather than licenses.

Typical enterprise RAG architecture

A reliable RAG rests on six building blocks. Ingestion pulls documents (SharePoint, OneDrive, DMS, Confluence, scanned PDFs) and keeps their access rights. Chunking splits each document into passages of 300 to 800 words with title, date and owner. Combined vector and keyword indexing clearly improves relevance compared with pure vector search. Permission filtering guarantees an employee never sees a passage they cannot access in SharePoint. Generation produces the answer with mandatory citations. Evaluation continuously measures quality on 150 to 300 reference business questions.

The most underestimated item is cleaning: out of 40,000 documents, 30 to 40 % are often duplicates, drafts or obsolete versions that must be excluded to avoid wrong answers.

What drives the price

Volume matters less than heterogeneity. 40,000 recent Word and PDF documents in SharePoint are processed faster than 10,000 documents spread across three DMS, scans and Excel sheets. Fine-grained access rights (by department, site, hierarchy level) add 15 to 25 % to the budget. Multilingual content (French and English) adds about 10 %. On inference, a company with 500 active users asking 5 questions a day typically uses 400 to 1,200 EUR per month with a high-performing model.

Mini case study

Nicolas, CIO of a 1,400-employee industrial company in Nantes, manages 40,000 procedure documents in SharePoint. An internal survey shows 300 employees lose about 1 hour a week looking for a procedure.

• Target gain: 50 % of that time, so 0.5 h x 45 weeks x 300 people = 6,750 hours per year.
• Average loaded hourly cost: 40 EUR, so 270,000 EUR of time recovered per year.
• Custom RAG project: 55,000 EUR excl. tax, plus 900 EUR per month for inference and hosting, so 65,800 EUR in year one.
• Copilot alternative for those 300 people: about 108,000 EUR per year in licenses.

Even if the estimated gain is divided by three, the project pays back in under 12 months.

FAQ

Can RAG make up answers?

The risk exists but drops sharply with mandatory citations and an instruction to answer « I don't know » when no source exists. On a well-built test set, the target is under 3 % ungrounded answers.

Do our documents leave the company?

Relevant passages are sent to the model through an enterprise API with no training, hosted in the EU. For the most sensitive data, a model deployed in your own cloud is possible, at an extra 500 to 2,500 EUR per month.

How is the 85 % correct-answer rate measured?

Your business experts write 150 to 300 questions with expected answers. The system is evaluated on this set before go-live, then every month.

Do we need to clean up SharePoint first?

Minimal sorting is enough: exclude archive folders and drafts. The project usually includes 2 to 4 weeks of assisted cleanup.

Can we start small?

Yes, with a pilot on one domain (HR or quality, for example) and 2,000 to 5,000 documents, for 12,000 to 20,000 EUR excl. tax in 6 to 8 weeks.

Let's scope your project. Describe your document sources, user count and access-rights constraints, and we will propose an architecture, an indicative budget and a 6 to 8 week pilot. Detailed quote within 48 h. WhatsApp +221 77 596 93 33.