AWS Lake Formation
A service that centralizes data lake construction, management, and security, providing column-level and row-level fine-grained access control for data on S3
Overview
AWS Lake Formation centrally manages the construction of S3-based data lakes, data ingestion, catalog registration, and access control. Integrated with the Glue Data Catalog, it provides data source ingestion (blueprints), automatic schema detection, table/column/row-level access control, and cross-account data sharing. When querying the data lake from Athena, Redshift Spectrum, or EMR, access permissions are managed through Lake Formation's unified policy framework.
Fine-Grained Access Control Beyond IAM Policies
Managing data lake access control solely through S3 IAM policies and Glue Data Catalog resource policies becomes complex as tables and users grow. Lake Formation provides its own permission model that grants access at the database, table, column, and row levels, managed through a grant and revoke mechanism similar to SQL GRANT/REVOKE. Column-level filtering lets you expose all columns to User A while showing User B only non-PII columns from the same table. Row-Level Security lets you define data filters for controls like "sales department users can only see their own department's data." Cell-level security combines column and row filtering, for example to restrict different columns depending on the row. These permissions apply to queries from integrated services such as Athena, Redshift Spectrum, and EMR.
Data Lake Construction Workflow and Blueprints
One workflow for building a data lake with Lake Formation consists of four steps: registering the data lake location, connecting data sources, ingesting data via blueprints, and configuring permissions. First, register an S3 bucket as the data lake location and delegate access permissions to Lake Formation. Next, configure connections to data sources (databases reachable over JDBC, such as RDS or on-premises databases). Blueprints are templates for common data ingestion patterns. The developer guide lists three types: database snapshot, incremental database, and log file (such as AWS CloudTrail and Elastic Load Balancing logs) (as of October 2026). When a blueprint runs, it automatically generates Glue crawlers and ETL jobs behind the scenes, extracting data from the source, storing it on S3 in the data format you specify, and registering it as tables in the Glue Data Catalog. The developer guide describes creating workflows from blueprints as simpler and more automated than creating them in AWS Glue. However, for complex data transformations or custom logic, creating Glue jobs directly offers more flexibility than blueprints.
Cross-Account Data Sharing and Governed Tables
Lake Formation's cross-account data sharing grants table-level and column-level data access permissions to other accounts within Organizations or external accounts. Traditional cross-account access via S3 bucket policies couldn't enforce controls based on logical data structures (tables, columns), but Lake Formation enables granular sharing like "Account B can read only columns A, B, and C of Table X." It integrates with AWS RAM (Resource Access Manager), standardizing the sharing invitation and acceptance workflow. Governed Tables are a special table type managed by Lake Formation that support ACID transactions, automatic data compaction, and time-travel queries. Even when multiple ETL jobs write to the same table simultaneously, transaction isolation levels are guaranteed, preventing readers from seeing partially-written data. According to the developer guide, Lake Formation's permission management also covers open table formats such as Apache Iceberg.
References (Official AWS Resources)
The primary sources for this page are the official AWS website and documentation. Check the official pages below for the latest specifications and pricing.
- AWS Glue official page
- Amazon Athena official page
- Amazon S3 official page
- AWS Documentation (official)
If this page and the official documentation disagree, treat the official documentation as authoritative.