Published
9 MIN READ
AI Agent With a Knowledge Base: What to Require
An AI agent with a knowledge base is an assistant that, before answering, searches for passages in the company's approved documents and writes the answer from them. It can only answer as well as that base allows. Before you switch the agent on, require an owner and an expiration date for each piece of content, a verified source, and permissions checked at the source.
In October 2026, the topic got a reference text from outside the world of tools. The Financial Stability Board's final report, published on October 15, 2026, calls for RAG procedures that confirm the retrieved and ingested data is secure, representative, and reliable. The text was written for financial institutions, but the bar works for any company that puts an agent in front of customers or staff.
At Wama, every AI output goes through human review before it reaches the client. With an agent that answers on its own, much of that review moves to the knowledge base: that is where the company decides what the agent can say. How the agent connects to the company's other systems is covered in the article on MCP for businesses; here the subject is the content it looks up.
What is an AI agent with a knowledge base?
It's a language model with a search step in front of it. A question comes in, a mechanism finds the most similar passages in the company's documents, and the model writes the answer using those passages as context. The technique is called retrieval-augmented generation, or RAG, and it doesn't change the model: it only changes what the model reads before answering.
This has a consequence few people read in the contract. The AWS security blog, in a post from September 18, 2025, treats the model as an untrusted entity, because it doesn't enforce authorization in its answers. The recommendation is to start from a simple premise: any data that enters the context can go back to whoever asked.
So the knowledge base is not a dead archive the agent consults discreetly. Everything in it that can be retrieved by that person can come out in an answer. That's why the right question isn't how many documents fit in the base, but which ones should be there. It's the same problem of who can see whose data, which comes up in the article on building a SaaS with AI.
Which documents go into the base, and which stay out?
In goes what the company would accept as an official answer: service descriptions, support rules, standard timelines, procedures, and FAQs the team has already answered. Out stays anything individual or sensitive: client contracts, personal data, credentials, financial spreadsheets, and drafts nobody approved.
The FSB report calls for a classification step that documents the sensitivity and the intended and prohibited uses of each piece of data. In practice, each document in the base gets a label: who it's for, whether it can be quoted to a customer, and what must never come out of it. Without that label, the decision is left to the search mechanism, which only measures text similarity.
When the data is personal, Article 6 of the LGPD, Brazil's data protection law, limits processing to the minimum necessary for the purpose. Uploading the entire support folder so the agent can answer questions about business hours already starts outside that bar. What the law requires of a product with AI is covered in the article on LGPD and artificial intelligence.
Who owns each answer in the base?
Each piece of content needs a responsible person and a review date. A knowledge base ages silently: prices change, timelines change, policies change, and the agent keeps repeating the old version with the same confidence. The FSB includes post-deployment monitoring to find problems caused by the data, such as data drift over time.
The worst case is a base with two versions of the same rule. The OWASP guide on vector and embedding weaknesses describes knowledge conflict: sources that contradict each other and a model that can't swap what it already knew for the new data. Deleting the old version is part of the update, not optional cleanup.
Ownership also applies to the account. In Wama's projects, the project is transferred to the client's account, and the client owns the website, the domain, and the account. The same logic works for the knowledge base: it belongs to the company, with admin access in the company's name, not to the personal account of whoever set up the agent. The rules for people who use AI day to day are in the AI usage policy.
Permissions: the agent sees what the person sees
The rule is simple to state and hard to follow: each person only gets from the agent what they could already open on their own. OWASP recommends fine-grained access control and permission-aware vector databases, with logical separation between user groups, because a shared base can leak context from one group to another.
The AWS post points out two flaws in the most common shortcut, metadata filtering. The vector database syncs periodically, so a permission revoked at the source keeps working in the agent until the next sync. And people who belong to hundreds of groups make the filter hard to get right. The recommendation is to check permissions at the source, at search time.
For the buyer, this becomes an objective question: if I remove someone's access right now, when does the agent stop showing that document to that person? The same logic of granting access in phases, applied to systems that write data, is in the guide to CRM integration with AI.
In a regulated company, the FSB also asks that the origin and transformations of the data be documented, what's called lineage, to make audits and problem fixing easier. The financial sector context is in the article on AI for fintechs.
What to require from the vendor before switching the agent on
Before you sign, ask in writing how each point below will be handled. The right-hand column shows the answer that should raise a red flag.
Point | What to ask | Red flag |
|---|---|---|
Selection | Which documents go in, and who approved the list? | Upload everything and let search decide |
Owner and expiration | Does each piece of content have an owner and a review date? | A base with no names and no dates |
Source | Where does each document come from, and who can add one? | Anyone can upload a file |
Permissions | Are permissions checked at the source, at search time? | Only a filter synced every so often |
No source | What does the agent do when it can't find the answer? | It answers anyway |
Logging | Is there a record of which passages were used in each answer? | Only the final text is saved |
Exit | If I switch vendors, do I take the base with me? | Base locked in the vendor's account |
At Wama, anything not written in the proposal is out of scope. With an AI agent, that rule matters even more: base maintenance, who reviews it, and how often need to be in the contract, or they end up with no owner. What the contract should say about ownership and documentation is in the article on custom software and who owns the code.
The request for logging has a technical basis. OWASP recommends detailed, immutable logs of retrievals, to detect and respond to suspicious behavior. Without them, when the agent gets something wrong, nobody can tell whether the problem was in the document, the search, or the model.
Common mistakes
Uploading the whole folder. Old documents, drafts, and personal spreadsheets become possible answers as soon as they enter the base.
Accepting outside files without checking them. OWASP describes a résumé with white text on a white background telling the system to recommend the candidate. The defense is to extract the text while ignoring formatting and to validate each document before it enters the base.
Treating poisoning as a lab topic. The full text of the FSB report notes that attackers can manipulate agents by injecting malicious data into the knowledge base, which influences their behavior over time.
Leaving the agent with no way out to a person. When the base has no source, the right answer is to say so and hand the conversation off. For fintechs serving consumers through an app, the SAC decree on fintech apps (Brazil's customer service rules) now requires the option to talk to a person.
Testing only with your own team's questions. Use the real questions that reach support, including the badly written ones, and repeat the test after every base update.
Frequently asked questions
Does an AI agent with a knowledge base need to train the model?
No. In most cases, the agent uses an off-the-shelf model and consults the company's documents on every question, through the technique called RAG. Training or fine-tuning a model is a different project, with a different cost and a different update routine. With a knowledge base, fixing an answer means fixing the document, without touching the model.
How often should the knowledge base be reviewed?
There's no universal timeline. The criterion is how fast the subject changes: prices and timelines need review with every change, while a service description can last months. What matters is that each piece of content has a review date and a responsible person, and that the old version comes out when the new one goes in.
Can the agent use customers' personal data?
Yes, when the purpose justifies it and access respects the permissions of whoever is asking. The LGPD limits processing to the minimum necessary, so personal data doesn't go into the general FAQ base. Individual data comes from the source system at question time, with the access check done there.
How do I know if the agent is making answers up?
Require each answer to show which document it came from and keep a record of which passages were retrieved. That way someone can compare the answer with the source. With no source, the agent should say it didn't find the information and hand the conversation to a person, instead of filling in with guesses.
Who should take care of the base once the agent goes live?
Whoever is responsible for the subject in the company, not whoever configured the tool. Support handles support answers, finance handles billing rules, and so on. The vendor can set up the process and train the team, but the decision about what is true stays inside the company.
If you're thinking about putting an agent in front of customers or staff, start with the list of documents and the owner of each one, before you choose the tool. To bring that list into your project, talk to Wama.
