
Key takeaways
- A chatbot that answers from your documents repeats whatever is in them, including the outdated price list nobody deleted.
- Duplicates are the most common problem. Keep one authoritative copy of each policy and archive the rest.
- Short, well-headed pages work better than long PDFs. Microsoft’s own guidance says to start each article with a summary and use consistent headings.
- Content needs an owner and a review date after launch, or accuracy drifts within months.
Why the documents matter more than the model
Most business chatbots don’t “know” your company. They search the files you give them, pull out the passages that look relevant and write an answer from those. That method is often called retrieval. It’s why a chatbot can quote your refund policy accurately on Monday and quote last year’s policy on Tuesday, if both versions are sitting in the same folder.
So before choosing a tool, look at what you’ll feed it. In our experience that’s where most of the work is, and it’s the part businesses tend to skip.
Step 1: Find and remove duplicates and old versions
Gather everything the bot might answer from: FAQs, policies, product sheets, help articles, onboarding guides. Then ask one question of each file: is this the current, official version?
Microsoft’s guidance for agents that answer from SharePoint is blunt about this. Keep only one authoritative copy of each document in the published library, and archive or delete duplicates. Labels like “current”, “archived” and “superseded” help if you can’t delete.
A quick way to spot trouble: search your files for your own prices, opening hours and delivery times. If the same fact shows up with two different values, the bot will eventually pick the wrong one.
Step 2: Restructure content so it can be found
Long documents with vague headings are hard for a retrieval system to slice correctly. Here’s what we’d change, based on Microsoft’s published recommendations for Copilot and its own knowledge-management advice:
| Problem | Fix |
|---|---|
| One 40-page PDF covering everything | Split it into short pages, one topic each |
| Headings like “General” or “Other” | Use descriptive headings that say what the section answers |
| Dense paragraphs of steps | Use numbered steps and bullet lists |
| Key facts inside images or scanned pages | Write the text out; Microsoft notes Copilot “cannot read text on images” |
| Complex tables | Keep tables simple, or turn them into headed sections |
| Links that say “click here” | Use descriptive link text |
Microsoft also suggests opening each article with a short summary stating its topic, purpose and audience. It’s a small habit that makes a big difference, because it helps the bot pick the right page and cite it.
Step 3: Take out what the bot shouldn’t see
Remove customer details, staff information and anything about suppliers that shouldn’t reach the public. Microsoft’s ingestion guidance lists exactly this: strip customer data and personal information about employees, vendors and others before content is ingested.
Permissions matter for internal assistants too. If a staff chatbot reads from a shared drive where everyone can see everything, it can surface an HR file to the wrong person. Tighten sharing before connecting it, not after. Data rules vary by region (GDPR in Europe, state laws in the US, PDPL laws in Saudi Arabia and the UAE), so check what applies to you.
Step 4: Fill the gaps with real questions
Your documents describe how things should work. Customers ask about how things actually go wrong. Pull the last few months of support emails, chat logs or call notes and list the 30 to 50 most common questions. Then check each one has a clear answer somewhere in your content.
Where there’s no good answer, write one. And decide which questions the bot should never handle alone, like complaints, refunds above a limit or anything medical or legal. Those need a route to a person, which we cover in our guide to chatbot human handoff.
Step 5: Keep it current after launch
A knowledge base isn’t a one-off project. Give every page an owner and a review date. Microsoft recommends setting an expiry date on each asset as a checkpoint to update or remove it, and using feedback such as thumbs-up ratings to spot pages that need work.
- Assign an owner for each topic area.
- Set review dates (quarterly works for most small businesses).
- Check the questions the bot couldn’t answer every week or two.
- Update the source document first, then let the bot re-read it.
If you’re starting from scratch, pick one narrow area, such as delivery and returns, and get that right before adding more. It’s the same logic we use when deciding which processes to automate first. If you’d like help cleaning up content and building the bot, see our AI chatbot development service or book a strategy call.
Frequently asked questions
What file types can an AI chatbot read?
How many documents do I need to start?
Can the chatbot learn from customer conversations on its own?
How often should I update the knowledge base?
Sources
- Microsoft Learn: Optimize SharePoint content for employee self-service agents
- Microsoft Dynamics 365 blog: Knowledge management best practices for Copilot ingestion (Jan 2024)
- Microsoft Learn: Add unstructured data as a knowledge source in Copilot Studio