Cost Anomaly Detection - Catching Unexpected Spending Early with AWS Cost Anomaly Detection

Why Cost Anomaly Detection Is Needed

In cloud cost management, threshold-based alerts from Budgets are effective, but they cannot catch every anomaly. For example, in an environment with a monthly budget of $10,000, if spending that is normally around $300 per day suddenly jumps to $800 one day, there are cases where you want to detect the problem before the end-of-month budget overrun alert fires. AWS Cost Anomaly Detection is a service that learns past cost patterns with an ML model and automatically detects spending fluctuations that differ from normal. Because it judges against a dynamic baseline rather than a fixed threshold, it enables highly accurate anomaly detection that takes into account seasonal variation (such as more batch processing at month-end) and trends (usage that increases gradually). Whereas Budgets notifies you based on a predefined threshold such as "80% of the budget has been reached," Cost Anomaly Detection notifies you based on a dynamic judgment such as "spending has deviated from the normal pattern." The two complement each other, and using them together is recommended.

Configuring Monitors and Subscriptions

Cost Anomaly Detection consists of two elements: monitors (what to watch) and subscriptions (how to notify). Monitors come in four types by the dimension they watch, namely AWS services, linked accounts, cost allocation tags, and cost categories. Each one is created with one of two methods, "AWS managed" or "customer managed." An AWS managed monitor automatically tracks the values within a dimension and maintains coverage without additional configuration even as new accounts, tag values, or categories are added. It evaluates up to 5,000 values per dimension individually (if there are more than 5,000 values, it tracks the top 5,000), and an AWS managed monitor is recommended for the main dimensions where you want to monitor the entire organization comprehensively. A customer managed monitor (formerly called a custom monitor), on the other hand, has you manually select up to 10 values to monitor (only one value for cost categories), and it combines the selected values into one group and evaluates their total. It is suited to cases where you want to vary the threshold for each value or where you want to monitor only a small number of specific accounts / teams. The AWS service monitor watches the costs of all services individually and detects anomalies in specific services. It is the most versatile and is the monitor you should set up first (AWS services are offered only with the AWS managed method). With the AWS managed method, the linked account monitor evaluates every member account in Organizations individually, and new accounts are also added to monitoring automatically (it can be created only in the management account). With the customer managed method, it evaluates the combined spending of up to 10 manually selected accounts, so you use it when you want to watch over a specific project or environment as a whole. The cost allocation tag monitor watches costs grouped by the values of a specific tag (Environment, Project, etc.), and the cost category monitor watches the costs of groups defined in Cost Categories.

# Create a service monitor
aws ce create-anomaly-monitor \
  --anomaly-monitor '{"MonitorName":"all-services","MonitorType":"DIMENSIONAL","MonitorDimension":"SERVICE"}' \
  --region us-east-1

# Create a subscription (notification)
aws ce create-anomaly-subscription \
  --anomaly-subscription '{
    "SubscriptionName": "cost-alerts",
    "MonitorArnList": ["arn:aws:ce::123:anomalymonitor/xxxx"],
    "Subscribers": [{"Type": "SNS", "Address": "arn:aws:sns:us-east-1:123:cost-anomaly"}],
    "Frequency": "DAILY",
    "ThresholdExpression": {"Dimensions": {"Key": "ANOMALY_TOTAL_IMPACT_ABSOLUTE", "Values": ["100"], "MatchOptions": ["GREATER_THAN_OR_EQUAL"]}}
  }' \
  --region us-east-1

In a subscription, you can set the notification frequency (immediate, daily, weekly) and an impact amount threshold (e.g., notify only for anomalies of $100 or more).

Root Cause Analysis and Response

When Cost Anomaly Detection detects an anomaly, it automatically runs a root cause analysis. The service, account, region, and usage type that caused the anomaly are identified and included in the notification. For example, specific information such as "RunInstances usage for EC2 in the ap-northeast-1 region increased to 3 times the normal level" is provided. The impact amount of the anomaly (the excess over the normal baseline) is also calculated, so you can judge the priority of your response. Through the integration with Cost Explorer, you can drill down into the cost breakdown for the period in which the anomaly was detected and check the details. Typical causes of anomalies include forgotten resources in development environments, unexpected scale-out by Auto Scaling, a surge in data transfer volume, charges from trying out new services, and resources created through unauthorized access. When an anomaly is detected, you first identify the cause, and you archive the anomaly if it is an intentional change (such as the release of a new feature) or deal with it if it is an unintended change.

Choosing Between It and Budgets, and Operations

Cost Anomaly Detection and Budgets complement each other, and using them together is the most effective. Budgets manages costs with an absolute threshold such as "notify if spending is likely to exceed $10,000 per month," preventing budget overruns. Cost Anomaly Detection uses a relative judgment such as "detect spending that deviates from the normal pattern" to catch unexpected fluctuations early. For example, suppose that in an environment with a monthly budget of $10,000, anomalous spending of $500 per day occurs at the beginning of the month. In that case, the Budgets 80% alert ($8,000) fires in the second half of the month, but Cost Anomaly Detection detects the anomaly and notifies you by the next day. As an operational best practice, we recommend starting by creating an AWS service monitor to watch all services and setting it to notify you daily of anomalies with an impact of $100 or more. Cost Anomaly Detection can be used completely free of charge, and you configure it through the Cost Explorer API (us-east-1 region).

Tuning to Reduce False Positives

To operate cost anomaly detection practically, it is important to tune it so that notifications are kept at an appropriate frequency. If the threshold is too low, alerts go off frequently even for normal fluctuations, and they eventually come to be ignored. Conversely, if it is too high, you miss real anomalies. By appropriately dividing what you monitor into units such as services, accounts, and cost categories, you can achieve highly accurate detection that takes the context of fluctuations into account. When a detection occurs, you look back on whether it was a real anomaly or an expected fluctuation and reflect that in a review of the settings. Continuously adjusting the detection accuracy and keeping a state in which only the spending fluctuations that truly need attention are notified is the key to keeping operations from becoming a mere formality.

Summary - Guidelines for Using Cost Anomaly Detection

AWS Cost Anomaly Detection is a service that discovers unexpected cost fluctuations early through ML-based dynamic anomaly detection. Because it detects deviations from past patterns rather than using fixed thresholds, it provides highly accurate alerts that take seasonal variation and trends into account. Its main strengths are cause identification through root cause analysis, immediate alerts through SNS notifications, and detailed analysis through the integration with Cost Explorer. It can be used completely free of charge and setup is completed in a few minutes, so we recommend enabling it in all AWS accounts. By using it together with Budgets, you can build a dual cost monitoring system that is both threshold-based and ML-based.