Building Enterprise Search with Amazon Kendra - Natural Language Queries and FAQ Auto-Extraction
Build an enterprise search platform that lets you search internal documents using natural language. This article covers data source connector configuration, search accuracy tuning, and RAG integration.
Overview of Kendra
Kendra is an ML-powered enterprise search service. Unlike traditional keyword search, when you ask a natural language question like "How do I apply for paid leave?", it extracts the relevant section from internal policy documents and provides a direct answer. Search results highlight relevant passages within documents. It connects to internal systems through over 40 data source connectors, supports natural language queries in 14 languages, and can also serve as a retriever for RAG. Internally, Kendra uses BERT-based transformer models for semantic matching between queries and documents, handling synonyms and variations without relying solely on exact keyword matches.
Data Sources and Accuracy Tuning
Data source connectors link to internal systems such as S3, SharePoint, Confluence, and ServiceNow, periodically crawling and indexing content. ACL-aware connectors filter search results based on user access permissions. For search accuracy tuning, registering custom synonym dictionaries, applying relevance boosting (weighting specific fields), and leveraging user feedback are effective approaches. Synonym dictionaries support both one-way mappings (e.g., "PTO" to "paid time off") and bidirectional synonyms, which helps cover industry jargon and internal abbreviations. Field boosting lets you assign a 10x score multiplier to titles or FAQ question fields, elevating precise matches above noisy body-text results.
RAG and Generative AI Integration
Kendra serves as a retriever for RAG (Retrieval-Augmented Generation), providing high-accuracy search results to generative AI applications. By combining Amazon Bedrock foundation models with Kendra, you can build chatbots that generate accurate answers based on internal documents. The Kendra Retrieve API fetches relevant document excerpts and passes them as context in the prompt to the foundation model. ACL-based access control ensures only results matching the user's permissions are returned, preventing confidential information leakage. Custom document enrichment lets you run Lambda functions to add metadata or preprocess text before indexing. For understanding Kendra's model design, related books (Amazon) can be a useful reference.
Comparison with Amazon Q Business
Amazon Q Business, which became generally available in 2024, is a fully managed service that incorporates Kendra's indexing technology while integrating generative AI answer generation and plugin connectivity. Q Business provides a one-stop chat interface experience for answering questions from internal information, eliminating the need to design a search UI from scratch. Kendra, on the other hand, excels in flexibility by allowing direct calls to the Retrieve API and Query API to embed search result ranking logic and presentation layers into your custom application. When you need to embed a search engine into an existing custom UI or portal, or when you want to select your own LLM and build a RAG pipeline, Kendra is the better fit. If you need to integrate actions like "book a meeting" or "create a ticket," Q Business's plugin mechanism has the advantage. The two services are not mutually exclusive - you can configure a Kendra index as a data source for Q Business.
Design Best Practices and Pitfalls
In index design, splitting each document into appropriately sized chunks significantly impacts search accuracy. Kendra processes up to 5 MB of text per document, but ingesting lengthy PDFs as-is can cause answers to lose focus. Pre-splitting by logical sections and attaching metadata (department, document type, publication date) to each chunk enables faceted search and filtering for improved accuracy. A common pitfall is misconfigured sync schedules. Setting connector sync to every hour triggers re-indexing even for unchanged documents, consuming query capacity units (QCUs) and inflating costs. For connectors with change detection, use incremental sync. For S3 connectors, triggering real-time sync via EventBridge object-creation events provides a more cost-effective approach. FAQ data sources accept CSV or JSON format Q&A pairs and display answers to common questions at the top of results, directly reducing support desk workload.
Kendra Pricing and Optimization
Kendra pricing depends on the index edition (Developer or Enterprise), connector sync frequency, and document count. The Developer edition costs approximately $810 per month and supports up to 10,000 documents and 4,000 queries per day. The Enterprise edition costs approximately $1,008 per month and supports up to 100,000 documents and 8,000 queries per day. You can scale with additional document storage and query capacity units. Optimize connector sync schedules to match data update frequency and avoid unnecessary re-indexing. Leveraging FAQ data sources to directly answer frequently asked questions improves both search accuracy and user experience. The Developer edition lacks high-availability architecture, so use Enterprise for production workloads and reserve Developer for PoC validation only.
Summary
Kendra is an ML-based enterprise search service that provides natural language question answering and serves as a retriever for RAG. It connects to internal systems through over 40 data source connectors and returns search results filtered by user permissions through ACL-based access control. Custom document enrichment and FAQ data sources further improve search accuracy. Understand the division of roles with Q Business, and choose Kendra when you need search UI customizability or want to build your own RAG pipeline.