Building a Cloud Data Warehouse with Amazon Redshift - Choosing Between Serverless and RA3
Learn the criteria for choosing between Serverless and RA3 provisioned clusters, and how to prevent data silos using data sharing and Spectrum for data lake integration.
Redshift Architecture Overview
Redshift is a cloud data warehouse built on columnar storage and a massively parallel processing (MPP) architecture. Columnar storage reads only the columns needed for analytical queries, dramatically reducing I/O compared to row-oriented RDBMS systems. The leader node handles query parsing and execution plan generation, while compute nodes perform parallel data processing. RA3 instances separate compute from storage, with data stored in S3-based Redshift Managed Storage (RMS). Frequently accessed data is cached on local SSDs, so you don't need to worry about read latency from S3.
Choosing Between Serverless and Provisioned Clusters
Redshift Serverless automatically scales capacity in RPU (Redshift Processing Unit) increments, and you incur no cost when queries aren't running. It's ideal for intermittent workloads such as periodic BI dashboard queries, ad-hoc analysis, and development/test environments. Provisioned clusters (RA3), on the other hand, are suited for always-on workloads. For production environments with 24/7 continuous query execution, combining RA3 with Reserved Instances can be more cost-effective than Serverless. As a rule of thumb, if daily query execution time is under 8 hours, Serverless is the better choice; if it exceeds 8 hours, RA3 provisioned clusters are more economical.
Data Sharing and Data Lake Integration with Spectrum
Data sharing is a feature that lets you share live data in real time between Redshift clusters. A producer cluster creates a data share, and a consumer cluster references it. No data copying occurs, and the consumer always sees the producer's latest data. This is effective when departments operate independent clusters but need to share common master data. Redshift Spectrum lets you run SQL queries directly on data stored in S3. Combined with columnar formats like Parquet or ORC, you can analyze petabyte-scale data lakes without loading data into Redshift. Using the Glue Data Catalog as a metadata store, both Redshift and Athena can query the same table definitions. For hands-on Redshift knowledge, you can also explore related books on Amazon.
Redshift Pricing
Redshift Serverless is billed based on RPU (Redshift Processing Unit) usage, with a base RPU starting at 8 and costing approximately $0.375 per RPU-hour. No charges apply when queries aren't running. For provisioned clusters, RA3.xlplus costs approximately $1.086 per node per hour (about $782/month), with Reserved Instance discounts of up to 64%. Redshift Managed Storage costs approximately $0.024 per GB per month. If daily query execution is under 8 hours, Serverless is more economical; for always-on workloads exceeding 8 hours, RA3 provisioned clusters are the better choice.
Summary
Redshift is a cloud data warehouse that delivers high-speed analytics on petabyte-scale data. A phased approach of starting small with Serverless and migrating to provisioned clusters as workloads grow is effective. By leveraging data sharing and Spectrum, you can prevent data silos and achieve unified analytics across your data lake.