AWS IoT Analytics Specialized2017年〜
A managed service for processing and analyzing large volumes of data collected from IoT devices
What It Does
AWS IoT Analytics is a managed service for collecting, processing, storing, and analyzing the massive volumes of message data sent from IoT devices. It automatically processes data filtering, transformation, and enrichment (joining with external data) through pipelines and stores the results in an analytics data store. You can run ad-hoc analysis with SQL queries or perform advanced analysis using Jupyter Notebooks.
Use Cases
Used for trend analysis of sensor data from factory production lines, optimization analysis of HVAC and lighting data in smart buildings, statistical processing of soil and weather data in agricultural IoT, and driving pattern analysis of vehicle telematics data.
Everyday Analogy
Think of it like a factory's quality control department. It receives inspection data from each stage of the production line, removes unnecessary data, cross-references it against baseline values, and generates analysis reports. It automates the work of extracting meaningful information from mountains of raw data.
What Is AWS IoT Analytics?
AWS IoT Analytics is a service for efficiently analyzing the large volumes of data generated by IoT devices. IoT devices send a wide variety of data at high frequency - temperature, humidity, vibration, location, and more. This raw data often contains noise and missing values, making it unsuitable for analysis as-is. IoT Analytics provides an end-to-end pipeline from data collection to cleansing, transformation, storage, and analysis, making it easy to put IoT data to work.
Data Processing with Pipelines
IoT Analytics pipelines let you apply multiple processing steps to messages received through channels (data entry points). You can remove unnecessary fields, convert data types, fill in missing values, apply custom transformations using Lambda functions, and join with external data from DynamoDB or S3 (enrichment). Processed data is stored in a data store and can be analyzed with SQL queries. Since data can be sent directly from IoT Core's rule engine, integration with devices is straightforward. For reference materials when working with pipeline data processing, related books on Amazon can also be useful.
Analysis and Visualization
Data stored in the data store can be extracted as datasets using SQL queries. Datasets support scheduled execution, enabling automatic generation of daily or hourly reports. Additionally, a managed Jupyter Notebook environment is integrated, allowing you to perform statistical analysis and build machine learning models using Python. Analysis results can be visualized on dashboards by integrating with QuickSight.
Things to Watch Out For
- Billed based on data processing volume, storage volume, and query execution. Be mindful of costs when storing large amounts of data long-term
- Not suited for real-time analytics. Consider using Kinesis Data Analytics if streaming analysis is needed
- Data can be lost if errors occur in pipeline Lambda functions. Implement proper error handling
