Machine Learning

Text Analytics and Natural Language Processing - Building an Intelligent Text Analysis Platform with Amazon Comprehend

Learn practical approaches to text analytics and natural language processing with Amazon Comprehend. Covers sentiment analysis, entity extraction, topic modeling, and custom model building with SageMaker integration.

約 6 分で読めます

Challenges in Text Analytics and an Overview of Amazon Comprehend

Approximately 80% of enterprise data consists of unstructured text, with vast amounts of information buried in customer reviews, support tickets, social media posts, and contracts. Amazon Comprehend is a fully managed natural language processing (NLP) service that uses machine learning to extract insights from text. It offers sentiment analysis, named entity recognition, key phrase extraction, language detection, and topic modeling, all accessible through simple API calls. With support for multiple languages including Japanese, it can be used for global text data analysis. Here is a CLI example of running entity recognition with Comprehend: ```bash aws comprehend detect-entities \ --text 'Sample Corp in Shibuya, Tokyo announced a new service in March 2026' \ --language-code en \ --region us-east-1 ```

Practical Applications of Sentiment Analysis and Entity Recognition

Comprehend's sentiment analysis classifies text into four categories - Positive, Negative, Neutral, and Mixed - and returns a confidence score for each. It can be applied to a wide range of use cases including automatic classification of customer reviews, brand reputation monitoring on social media, and priority assessment of support tickets. Entity recognition automatically extracts named entities such as person names, organization names, locations, dates, and quantities from text. This directly supports business process automation, from extracting party names from contracts to identifying company names and amounts in news articles, and detecting drug names and symptoms in medical documents. The PII (Personally Identifiable Information) detection feature automatically identifies phone numbers, email addresses, credit card numbers, and other personal information in text, enabling masking and redaction workflows.

Custom Classification and Custom Entity Recognition

Comprehend's custom classification feature lets you build text classification models based on industry-specific category systems. Simply upload a CSV file of pre-classified text to S3, and Comprehend automatically trains the model and deploys it as an endpoint. Custom entity recognition lets you build models that recognize industry-specific terms not covered by standard entity types, such as product names, internal codes, and specialized terminology. It offers two training modes - annotation mode and entity list mode - so you can choose based on your data readiness. Integration with SageMaker allows you to further fine-tune Comprehend custom models or pass Comprehend output to downstream SageMaker pipelines for additional analysis. The Flywheel feature automates the continuous improvement cycle, automatically retraining models as new data accumulates to improve accuracy over time. For a systematic study of text mining from basics to advanced topics, check out books on Amazon.

Batch and Real-Time Analysis Architectures

Comprehend offers two processing modes: batch analysis and real-time analysis. Batch analysis asynchronously processes large volumes of text data stored in S3 and outputs results to S3. It is ideal for processing large datasets, such as bulk sentiment analysis of millions of customer reviews or topic classification of historical support tickets. Real-time analysis returns results instantly through API endpoints, making it suitable for chatbot intent classification and real-time content moderation. A serverless architecture combining API Gateway and Lambda provides automatic scaling and cost optimization based on request volume. Integration with Kinesis Data Streams enables building real-time analysis pipelines for streaming data. By storing analysis results in DynamoDB or OpenSearch and visualizing them on dashboards, you can share text data insights across the entire organization.

Comprehend Pricing

Comprehend pricing is based on the volume of text processed. Sentiment analysis, entity extraction, and key phrase extraction cost approximately $0.0001 per unit (100 characters). Custom classification model training costs approximately $0.0005 per second, and inference costs approximately $0.0005 per unit. PII detection costs approximately $0.0001 per unit. For large text volumes, the asynchronous batch API is more cost-effective than the synchronous API. The free tier includes 50,000 units per month for each API during the first 12 months.

Summary - Guidelines for Building a Text Analytics Platform

Amazon Comprehend provides fully managed text analytics and natural language processing, enabling high-accuracy text analysis without requiring machine learning expertise. In addition to standard features like sentiment analysis, entity recognition, and PII detection, you can build custom models tailored to industry-specific category systems and specialized terminology. With advanced model tuning through SageMaker integration, batch analysis of large text volumes in S3, and real-time analysis via API Gateway and Lambda, it covers a wide range of use cases from large-scale data processing to real-time content analysis.

Related Services

Amazon ComprehendA service that uses natural language processing to extract entities, sentiment, and key phrases from textAmazon SageMakerAn integrated platform for building, training, and deploying machine learning models

Related Articles

Data Search and Analytics in Practice - Building Full-Text Search and Visualization with OpenSearchLearn about data search and analytics design with Amazon OpenSearch Service, including how to build an analytics foundation with full-text search, log analysis, and dashboard visualization.Designing Streaming Data Processing - Building Real-Time Data Pipelines with KinesisLearn streaming data processing design techniques with Amazon Kinesis, including building real-time data pipelines with Data Streams, Data Firehose, and Lambda integration.Document Text Extraction - Intelligent Document Processing with Amazon TextractLearn how to automatically extract text, tables, and form data from documents with Amazon Textract, and build natural language processing pipelines by integrating with Amazon Comprehend. This article covers automation patterns for invoice processing and contract analysis.

More on This Topic

GPU-Based Machine Learning Training with AWS Batch - Cost-Efficient Large-Scale TrainingRun GPU training with your existing Docker containers, and cut costs by up to 90% using Spot Instances and checkpointing. Includes guidance on when to choose Batch over SageMaker.Using Claude on Amazon Bedrock - Model Selection, Prompt Design, and Cost OptimizationCompares the Anthropic Claude models available on Amazon Bedrock, provides model selection guidelines by use case, and covers prompt design best practices and cost optimization.Building RAG Applications with Amazon Bedrock Knowledge Bases - Implementing Retrieval-Augmented GenerationAutomatically index documents on S3 and unify search and generation with the RetrieveAndGenerate API. Covers chunking strategy selection and safety enforcement with Guardrails.Getting Started with Quantum Computing on Amazon Braket - Designing and Simulating Quantum CircuitsPrototype for free with local simulators, then run quantum circuits on IonQ and Rigetti hardware. Covers implementing VQE and QAOA with hybrid jobs.Privacy-Preserving ML with AWS Clean Rooms ML - Build Models Without Sharing DataLearn how to build lookalike models with Clean Rooms ML, apply differential privacy, and leverage the results for ad targeting.Implementing Natural Language Processing with Amazon Comprehend - Sentiment Analysis and Entity ExtractionLearn about sentiment analysis, entity extraction, and building custom classification models with Comprehend.Building Conversational Bots - Natural Conversation Interfaces with Amazon Lex and PollyLearn how to build conversational bots using Amazon Lex and Amazon Polly.Demand Forecasting - Predicting the Future from Time Series Data with Amazon ForecastInput historical time series data and related variables to automatically build ML-based demand forecasting models. This guide covers forecast accuracy evaluation metrics and patterns for leveraging forecast results through S3 and QuickSight integration.

Similar Articles and Services

Implementing Natural Language Processing with Amazon Comprehend - Sentiment Analysis and Entity ExtractionLearn about sentiment analysis, entity extraction, and building custom classification models with Comprehend.Document Text Extraction - Intelligent Document Processing with Amazon TextractLearn how to automatically extract text, tables, and form data from documents with Amazon Textract, and build natural language processing pipelines by integrating with Amazon Comprehend. This article covers automation patterns for invoice processing and contract analysis.Amazon TranscribeA high-accuracy speech recognition service that automatically converts audio to textNo-Code ML with Amazon SageMaker Canvas - Building Prediction Models with a Visual InterfaceA visual tool for building ML models without writing code. Simply upload a CSV and select a prediction target to build a model, with bidirectional sharing with Studio.