AWS Storage Gateway
A hybrid cloud storage service that transparently provides S3, FSx, and EBS storage to on-premises applications as file shares, volumes, and tape libraries
Overview
AWS Storage Gateway is a hybrid storage service that seamlessly connects on-premises IT environments with AWS cloud storage. Deployed as an on-premises VM or hardware appliance, it provides three interfaces: NFS/SMB file shares, iSCSI block volumes, and virtual tape libraries. Frequently accessed data is kept in a local cache, while all data is automatically backed up to AWS storage services.
Four Gateway Types and Use Cases
Storage Gateway offers four types: S3 File Gateway, FSx File Gateway, Volume Gateway, and Tape Gateway. S3 File Gateway receives files via NFS/SMB protocols and stores them as objects in S3 buckets. On-premises applications see it as a standard file share, while behind the scenes it leverages S3's durability and scalability. FSx File Gateway provides locally cached access to FSx for Windows File Server, suited for hybrid Windows file share configurations requiring Active Directory integration. Volume Gateway has two modes - cached and stored - providing block storage via the iSCSI protocol. Cached mode stores all data in S3 with only hot data cached locally, while stored mode keeps all data locally and asynchronously backs it up to S3. Tape Gateway operates as a virtual tape library (VTL) and can be used as a tape backup replacement with existing backup software such as Veeam, Commvault, and Veritas.
Cache Design and Bandwidth Throttling
Storage Gateway performance depends heavily on local cache design. The cache disk must be sized to accommodate the working set (frequently accessed data) - when cache hit rates drop, all access traverses the WAN, causing a sharp increase in latency. Monitor the CloudWatch CacheHitPercent metric and consider adding cache disk capacity if it falls below 80%. For S3 File Gateway, the minimum cache disk size is 150 GB and the maximum is 64 TB. Bandwidth throttling lets you cap the network bandwidth consumed by the gateway. You can configure schedules that limit bandwidth during business hours and allow full bandwidth for data uploads overnight. The upload buffer disk temporarily stores data awaiting upload to S3 - in bandwidth-constrained environments, insufficient buffer sizing causes write errors due to buffer overflow. For a comprehensive look at hybrid cloud storage strategies, related books (Amazon) are a helpful resource.
VM Deployment and Hardware Appliance
The most common deployment method for Storage Gateway is as a virtual machine on VMware ESXi, Microsoft Hyper-V, or KVM. Minimum VM requirements vary by gateway type, but for S3 File Gateway the minimum configuration is 4 vCPU, 16 GB RAM, and 150 GB of cache disk. Production environments recommend 8 vCPU and 32 GB RAM or higher. For environments without virtualization infrastructure or where hardware management simplification is desired, AWS offers a hardware appliance. Delivered as a 1U rack-mount server with gateway software pre-installed, it ships ready to activate. Deploying a gateway on an EC2 instance is also possible, used for patterns such as accessing files via S3 File Gateway from AWS workloads or volume recovery in DR environments. Gateway updates are automatically applied within the maintenance window, typically completing with just a few minutes of downtime. For high availability requirements, place the gateway VM on a VMware HA cluster to configure automatic failover on host failure.