AI knowledge assistant for business (RAG)

An internal AI knowledge assistant answers employee questions from company sources and shows a path back to the source. RAG, short for retrieval-augmented generation, describes the sequence: search the permitted source set, select relevant passages, and give those passages to a language model to compose an answer.

Decide whether that sequence solves a problem that search, an FAQ, or a structured tool cannot. Start with the questions employees ask and the records they need. The chat interface is the last part of the decision.

Start with the question employees need answered

Collect a reviewed set of real questions before selecting a vendor. Include the expected answer, source, access scope, and acceptable refusal. If the questions are vague or the source owner is unknown, the first project is information stewardship rather than an assistant.

What RAG adds

RAG can connect an employee's wording to current passages in the company's source set. It can expose a citation and decline to answer when no permitted passage supports the request. Those properties make a response easier to check than an answer generated from the model's general training alone.

Each one requires deliberate design. A RAG system can retrieve an outdated policy, select a paragraph without its heading, expose a document from the wrong access scope, or cite a passage that does not support the conclusion. The acceptance plan must test retrieval, source status, citation correctness, and refusal behavior separately.

Retrieval before generation

The retrieval path should be visible during review. For each question, record the query, candidate passages, rank, source version, access decision, selected context, and citation in the answer.

Exact identifiers often need a lexical search path. A policy question phrased as a paraphrase may benefit from vector similarity. A hybrid design can use both signals and then rerank a short candidate list. Anthropic's Contextual Retrieval article describes adding document context to chunks and combining semantic and lexical retrieval with optional reranking. Treat that as a design reference and measure on your own corpus.

Chunking is part of retrieval quality. Keep enough heading and document metadata for a passage to make sense, preserve source dates and links, and avoid mixing current and superseded text in one context. If the source set is small and stable, a full-text index may be the better system.

Sources need owners

The assistant cannot correct a source that nobody maintains. For each document set, assign an owner and record its status, effective date, superseded version, access scope, and update path. Give the user enough information to judge whether a citation is current.

When two current sources disagree, return a conflict state and route it to the owner. Do not ask the model to average the statements or choose the newer file by a hidden rule. The conflict needs a resolution, a person responsible, and a new source status.

Source preparation can include removing duplicates, adding text layers to scans, separating versions, and agreeing names for fields and records. Include that work in the scope because useful retrieval depends on it before launch.

Permissions before the model

Access filtering must happen before a passage enters the model context. A generated answer cannot reliably remove information the model has already seen. Test the same question with different roles, a mixed-permission source set, a revoked document, and a user whose access changed.

Log the access decision without exposing restricted content to someone who cannot inspect it. Define retention and audit access for queries, retrieved passages, answers, and corrections. The GDPR text is the primary legal source for personal-data obligations; the deployment still needs an organisation-specific assessment.

Test on a question set

Use real questions and representative source files. Include:

• paraphrases and exact identifiers;
• answers split across sections or files;
• missing and stale information;
• conflicting current sources;
• scanned or poorly structured documents;
• permission-restricted passages;
• prompt-injection text embedded in a source; and
• requests that should become a tool call rather than an answer.

For each case, measure retrieval recall, source and citation correctness, refusal correctness, permission enforcement, field validity where relevant, and latency. Keep the test set stable enough to compare a parser, chunking, embedding, or reranking change. NIST's AI Risk Management Framework recommends representative testing and monitoring over the system lifecycle.

Price and fit

Scope follows the work the assistant must own. A single, clean source and one internal channel can be a small automation. Several systems, role-based access, scans, source cleanup, connectors, monitoring, and approval flows create a larger build and an ongoing operating responsibility.

Ask for those lines separately in a proposal. The AI apps service page describes the category, while the pricing page gives current commercial structure. The useful comparison is not a model name or a chat screenshot. It is the question set, source ownership, access boundary, acceptance checks, and maintenance plan.

When search or FAQ wins

Choose search or an FAQ when the source set is small, stable, and easy to navigate. Choose a RAG assistant when questions span maintained sources, employees need citations, and permissions must follow the person asking. Choose a workflow when the result must read or change another system.

A RAG assistant is a poor first project when documents contradict one another, there is no owner, or nobody can define what a correct answer looks like. Fix those conditions or choose a narrower source set before adding a model.

Operating ownership

Assign owners for the source set, access policy, question set, evaluation, and incident response. Review a sample after source changes and after changes to prompts, retrieval, model, or connectors. Keep the answer, citation, source version, and approval event linked when the result affects a business record.

The Company Brain AI service is a broader product question than a document assistant. It becomes relevant when the system must retain approved decisions, distinguish current state from history, or accept work results back into shared memory. Do not choose that broader scope until the first question set shows that a document assistant is insufficient.

FAQ

Does RAG eliminate unsupported answers? No. It supplies evidence candidates and a source path. Retrieval, source status, permissions, generation, and review still need controls.

Can we start with a public chatbot? An internal assistant and a customer-facing chatbot have different source, access, and escalation requirements. Decide who may ask questions and what the answer may trigger before selecting a channel.

What is the first useful deliverable? A reviewed question set with expected sources, access scope, and acceptable refusal. It makes a vendor discussion and a technical test concrete.

Sources

• Anthropic: Contextual Retrieval
• NIST AI Risk Management Framework 1.0
• GDPR, Regulation (EU) 2016/679