Website AI assistants: how RAG works and where to start

A website AI assistant should answer from reliable company information, point to supporting sources and hand difficult situations to a person. RAG, or retrieval-augmented generation, is one approach: the application retrieves relevant material before asking a model to formulate an answer. The quality of the result begins with knowledge management and operating rules.

An ambitious brief may describe an assistant that understands every product, advises customers, creates orders and resolves disputes. A practical first step is narrower: explain delivery conditions, find an instruction or identify the right service. One thoroughly evaluated journey offers a stronger foundation than many loosely defined capabilities.

How retrieval supports an answer

The system uses the visitor's question to search permitted sources, then supplies selected passages to the model. The answer can include links to those documents. The AWS documentation on knowledge bases and RAG describes this use of retrieved company information and citations.

Imagine a customer asking about delivery of an oversized item. The assistant needs the applicable delivery policy, not a general rule for small parcels. It may need to ask for a destination. When the available material does not establish an answer, a specific clarification or a handoff is a successful outcome.

Build a usable knowledge base first

Collect documents that support staff actually use: service conditions, instructions, model references and answers to recurring questions. Give each source an owner, review date and scope. Two conflicting policies do not become consistent because they have been uploaded to an AI system.

• Public information may be used for any visitor.

• Customer-specific records require identity and record-level access checks.

• Internal documents must remain outside a public assistant's permitted sources.

• Live information such as stock or order status needs a current source rather than an old document copy.

The Service in UA case shows a repair journey organised by device, model and service. It is a useful illustration of structured service information: an answer depends on the right combination of attributes, not just a matching keyword in a long document.

Evaluate retrieval separately from wording

A fluent response based on instructions for the wrong model is still wrong. Check which documents were found before judging how naturally the answer reads. Include similar names, abbreviations, spelling mistakes, different languages and questions for which the knowledge base has no answer.

A precise product code may require an exact match. A description of a problem in everyday language may benefit from semantic retrieval. Compare the approaches on real tasks instead of assuming that one method should replace all others. Our guide to ecommerce search and filtering explores these different intentions.

Check citations as well. A valid URL does not prove that the linked page supports the claim. The selected passage should actually justify the answer, and the customer should be able to open the relevant source. If the document is private, the interface must respect that boundary too.

Define what the assistant is allowed to do

Explaining a return policy, drafting a request and approving a refund have very different consequences. The first may rely on public information. The second needs validated fields and the user's agreement. The last must pass the company's authorisation and business rules, regardless of what the conversation says.

OWASP describes prompt injection through user input or external content and notes that RAG does not remove the risk. Restrict privileges and enforce consequential operations in application code. See OWASP's prompt injection guidance.

Rather than giving an assistant general CRM access, define narrow operations such as reading the status of an authorised order or submitting a support request. The server should validate identity, permissions and parameters on each call. A statement in a conversation is not evidence that the speaker owns a particular record.

Design a measurable pilot

1. Collect representative support questions and remove unnecessary private information.

2. Record expected facts, supporting sources and cases requiring a person.

3. Test document retrieval, factual correctness and citation support separately.

4. Include missing data, contradictory policies and attempts to override the assistant's rules.

5. Compare the result with ordinary search or a well-designed help form.

Measure supported answers, confidently incorrect answers, successful handoffs, waiting time and cost per completed journey. The number of chat messages is not a useful success metric on its own. A long conversation may mean the customer never found the answer.

Plan for ongoing operation

The budget includes more than model requests. Documents need maintenance, indexes need updating, failures need monitoring and somebody must own the knowledge base. Costs depend on traffic, context size, integrations and availability requirements. A measured pilot provides a better estimate than a price based on the appearance of the chat widget.

Define the fallback when the model is unavailable or a budget limit is reached. Visitors should still be able to search, contact the company or reach a member of staff. Also establish a correction process: who changes an inaccurate policy, how it reaches the index and how the team confirms that answers now use the current version.

Write a problem statement rather than an AI wish list

Describe where people lose time: support repeats the same explanation, visitors cannot find a relevant instruction or partners struggle to locate a document. Add example questions, available sources and clear responsibility limits. The team can then compare a conversational assistant with navigation improvements or a structured form.

Development may itself use AI tools, but that does not replace product evaluation. Our article on AI-assisted web development explains the distinction. A focused pilot can be considered as part of web application development once the underlying problem is clear.

Frequently asked questions

Does RAG guarantee that answers will never be invented?

No. Retrieval adds evidence, but quality still depends on sources, selection and evaluation. The assistant needs a reliable way to acknowledge insufficient information.

Must we train our own model immediately?

Not necessarily. First test access to current knowledge and the core journey. Model training and document retrieval address different needs.

When might a chatbot be unnecessary?

When the customer simply needs a price, a filter or one clear instruction. Improving the page itself may be the most effective first step.