An owner who wants an AI assistant plugged into the company's documents almost always starts with the same question: which model should we use? It is the least useful question in the project. When we build AI agents for small businesses, the failure we see most often comes from a model that reasons very well, on the wrong version of the procedure.
An assistant of this kind first searches your documents for the passages that seem to answer the question, then writes its answer from those passages. The model only sees what the search hands it: if your folders hold three versions of the warranty policy, it will answer confidently from whichever one it found.
This article covers the work that decides answer quality, none of it technical, and then what a ChatGPT, Copilot or Claude subscription already does on its own.
The short answer, for the busy
An AI assistant that answers from a company's documents works in two steps: it searches your files for the relevant passages, then writes an answer from what it found. Answer quality therefore depends on your documents before it depends on the model. Four decisions do most of the work: one current version of each procedure, an owner and a review date for every document, answers that cite the document and passage they used, and a set of test questions written before launch and re-run after every change. For a small, stable set of documents that the whole team is allowed to read, a ChatGPT or Claude project, or Copilot inside Microsoft 365, is often enough. A built assistant earns its keep when the documents run into the hundreds across several systems, when access rights differ from one person to the next, when the answer depends on live data, or when you need a log of questions and answers.
Search, then answer: the mechanism without the jargon
Picture a gifted researcher locked in a room who only reads the pages a clerk slides under the door. The researcher writes fast and well, but has no way of knowing that one page dates from 2019, or that another version sits in a different filing cabinet. The clerk is the search step; the researcher is the model.
Vendors describe this themselves. In a Claude project on a paid plan, when the documents approach the limit of what the model can read in one go, Claude automatically switches to a search tool: instead of loading everything, it fetches the relevant passages. Anthropic says this mode expands a project's capacity by up to 10x. One of its tips for making that search work is almost mundane: clear, descriptive file names.
A stronger model writes better, and sometimes searches better. It cannot guess which version of a procedure is the real one if nothing in your files says so. That problem gets fixed in your folders.
The real work: document hygiene
The first problem is contradiction. In almost every small business, the same procedure exists in several copies: the employee handbook version, the one emailed around last year, the file called "returns_procedure_v3_FINAL" and its edited copy. An experienced employee knows which one everyone follows. The assistant sees conflicting texts with equal authority. Keep one, and move the others out of its reach.
The second is the missing owner. A document nobody owns ages quietly. Every document the assistant can read needs a named owner: the person who fixes it when reality changes, and who hears about it when an answer goes wrong.
The third is the missing date. Put a review date on each document and a deadline for the next review, shorter for a price list than for a policy that rarely moves. A document past its deadline leaves the collection until its owner has re-read it.
The fourth is what you leave out. More documents do not produce better answers. Drafts, email threads, meeting notes where a decision was discussed and then reversed, old price sheets: each one gives the search a chance to bring back a passage that is plausible and wrong. Anthropic advises uploading all relevant documents to a project up front, and that is good advice for a collection that has already been sorted. Applied to a shared drive as it stands, it does the opposite. As for personal information, keeping it out is an architecture question, covered in our article on confidential data.
All of this fits in a four-column table: document, owner, review date, status (current or archived). It is the most useful deliverable of the project, and it helps the employees who will never open the assistant.
An answer that cites its source
An assistant that answers without saying where the answer came from asks staff to trust it. One that shows the document name and the passage it used lets them check in ten seconds, before the answer goes to a customer.
The source also changes how the tool improves. When a wrong answer cites the 2021 policy, the problem is located: a document to fix or archive, an owner to tell. Without a source, a bad answer becomes a mystery that people try to solve by rewriting the instructions, which does nothing for an outdated document. This is the failure we called "stale knowledge" in our article on hallucinations: the answer looks right, and the source is old.
And when the search brings back nothing solid, the assistant should say so and name the person to ask, rather than filling the gap with general knowledge.
Test questions, written before launch
Before connecting anything, ask the people who will use the assistant to write down the questions they actually ask, in their own words, abbreviations and typos included. Thirty to fifty is enough to start. For each one, note the expected answer and the document that contains it. Add a few questions whose answer is in no document: the right answer is then "I don't know," followed by the name of the person to ask.
That list becomes your measuring instrument. Run it at launch, then after every change: a batch of new documents, a rewritten procedure, a new model or new instructions. Twenty new documents can make a question that worked yesterday pull up a passage from an unrelated file, and only re-running the whole list shows it. You don't need a success rate published by a vendor; you need yours, measured on your questions.
A useful side effect: while finding the document behind each expected answer, you discover the frequent questions that no document answers. How AI agents are built explains why this evaluation is prepared before going live.
When ChatGPT, Copilot or Claude is enough
Before building, look at what your subscription already does. All three main tools let you gather documents in a workspace and ask questions about them.
ChatGPT. A project groups files, instructions and chats. On ChatGPT Business, a project takes up to 40 files and up to 100 collaborators, and you can add links to Google Drive files or folders and Slack channels as sources. By default, OpenAI does not train its models on content from a Business workspace.
Claude. Each project file can be up to 30 MB, and on paid plans the search mode described above takes over as the volume grows. On Team and Enterprise plans, a project can be shared with colleagues, with view or edit access.
Copilot. Without the paid Microsoft 365 Copilot license, Copilot Chat cannot reach company data, but anyone can upload a file to the chat, and the file is stored in their OneDrive. With the license, Microsoft's Agent Builder lets you describe, in plain language, an assistant that answers from a team's SharePoint files. Microsoft states that these agents respect existing permissions: an employee does not see content from a site they cannot access.
These tools are enough when the collection is small and stable, everyone on the team may read all of it, and a person reviews each answer before acting on it. To choose the subscription itself, see our comparison of Copilot, ChatGPT and Claude. The cleanup doesn't change: a ChatGPT project full of contradictory versions will answer as badly as a custom assistant.
When a built assistant is worth it
Four signals tip the decision.
Volume and sprawl. When documents number in the hundreds and live in several systems (a shared drive, a wiki, line-of-business software, mailboxes), dropping them into a project by hand becomes a chore nobody repeats. A built assistant reads them at the source.
Per-person permissions. In a ChatGPT or Claude project, anyone with access to the project sees all of its files. As soon as access depends on the role (HR files, margins, a client's contracts), the assistant has to know who is asking and search only the files that person can already open. Copilot does this for Microsoft 365 content; elsewhere, it has to be designed.
Live data. "Has this customer paid their last invoice?" or "how many are left in stock?" is in no document. The answer comes from software, at the moment of the question, through an API integration.
The log. To know who asked what and what the assistant answered, you need a log you control. An assistant that answers customers rather than employees always needs one, which is the case covered in our article on AI agents in customer service.
Where to start
Pick one area, the one that generates the most repeated questions: shop-floor procedures, HR policies, product sheets. Build the document table, with an owner and a date for each. Write thirty test questions. Then try it with the subscription you already pay for.
If the test passes, you have your assistant, and you don't need us. If it runs into volume, permissions or live data, let's talk: we start from your table and your test questions, which you will need either way.
→ First conversation, no commitment
Product features and limits were read on the OpenAI, Anthropic and Microsoft help pages on October 15, 2026; they change often.
Written byXavier Peich