Applies to Product - Microsoft Copilot Studio
What’s happening?
Customers are experiencing issues with the quality of output from a knowledge base that includes a large number of PDF files (over 9000), each containing approximately 50 to 350 pages. The responses generated, particularly regarding date-related queries, are often inadequate, too brief, or incorrect.
Reason:
There is a limit: An Agent can include up to 500 files as knowledge, here is the public documentation: Upload files as a knowledge source - Microsoft Copilot Studio | Microsoft Learn. The underlying cause of the issue appears to be related to the handling of a large volume of files and the format in which they are stored. Specifically, when using JSON format for metadata retrieval, the bot may not accurately filter or respond to queries based on dates or specific content within the PDF files.
Resolution:
There is a limit: An Agent can include up to 500 files as knowledge, here is the public documentation: Upload files as a knowledge source - Microsoft Copilot Studio | Microsoft Learn.
- Test the bot's behavior using a smaller number of files (e.g., 10) to determine if the output quality improves.
- Ensure there are no duplicate files in the knowledge base, as files with the same content may confuse the bot and lead to inaccuracies.
- Consider removing JSON files from the knowledge source and using the PDF files directly from SharePoint, as they may provide more accurate responses.
- Verify that the bot's instructions prioritize user-uploaded documents over pre-configured knowledge sources.