Building a Managed SFTP Server with AWS Transfer Family - S3 Integration and User Management
Learn how to build SFTP/FTPS servers with Transfer Family, integrate with S3, and configure custom authentication.
Overview of Transfer Family
Transfer Family is a service that provides managed servers supporting four protocols: SFTP, FTPS, FTP, and AS2. If you use SFTP for file exchange with trading partners, you can build an SFTP server with Transfer Family and set S3 as the transfer destination. Existing SFTP clients and scripts can continue to be used by changing the connection hostname. EFS (Elastic File System) can also be selected as the backend storage for workloads requiring POSIX permissions.
Authentication and Workflows
User authentication can be configured using one of three methods: service-managed (managed within Transfer Family), custom (Lambda authorizer), or AWS Directory Service. Custom authentication lets you implement authentication logic in a Lambda function and integrate with existing LDAP or Active Directory. The Lambda authorizer response can dynamically determine per-user home directories, IAM roles, and S3 bucket prefixes, supporting multi-tenant configurations. Managed workflows are processing chains that run automatically after file uploads, allowing you to define file copying, tagging, PGP decryption, and custom Lambda processing.
AS2 Protocol and Automation
The AS2 (Applicability Statement 2) protocol is used for B2B file exchange, providing message encryption, signing, and delivery receipts (MDN). Transfer Family's AS2 connector lets you set up automated file exchange with partners using S3 as storage. Managed workflows execute automated processing (copying, tagging, Lambda invocation) after file uploads, building file processing pipelines. CloudWatch Logs records transfer logs for auditing file send and receive activity.
Design Best Practices and Pitfalls
Endpoint type is decided at the start of the design. Public endpoints do not provide fixed IP addresses, which makes them harder to handle in partner firewall allowlists. VPC endpoints allow fixed IP assignment via Elastic IP, enabling IP notification to partners. VPC-internal endpoints restrict access to VPN/Direct Connect connections and do not accept connections from the internet. Endpoints are charged by the hour while a protocol is enabled. For small-scale use cases with transfers about once a day, one approach is to compare costs with S3 presigned URLs or DataSync on-demand transfers. Also, conflicts between S3 bucket policies and user IAM role permissions sometimes cause unexpected access denials. Grant users' IAM roles minimum permissions to the S3 bucket only, and explicitly allow access from the Transfer Family service role in the bucket policy.
Comparison with Alternative Services
File transfer methods compared with Transfer Family include S3 presigned URLs, DataSync, and Storage Gateway. S3 presigned URLs require no always-on server, but they require partners to change their SFTP clients (migrating to URL-based transfers), so existing scripts cannot be used as they are. DataSync is used for bulk data migration from on-premises and offers bandwidth control and scheduling, but does not support inbound transfers (push-type) from partners. Storage Gateway (File Gateway) provides NFS/SMB mount points on-premises but is not suited for exposing to external partners. With Transfer Family, existing SFTP workflows can be kept as they are while migrating to the cloud. FTP does not encrypt communications, so one approach for new deployments is to use SFTP or FTPS.
Transfer Family Pricing
Transfer Family pricing is based on the time each protocol is enabled on a server endpoint and the amount of data uploaded and downloaded, among other factors. Since the always-on endpoint cost is the primary expense, evaluate cost efficiency in advance if transfer frequency is low. Endpoint fees apply per protocol, so enabling SFTP and FTPS on the same server incurs fees for 2 protocols. S3 storage costs are billed separately. In managed workflows, charges apply to the amount of data processed in the PGP decryption step. AS2 is billed by the number of messages sent and received, in addition to the time the protocol is enabled.
Summary
Transfer Family provides managed servers for SFTP, FTPS, FTP, and AS2 protocols, simplifying file storage management through S3 integration. Custom authentication uses Lambda authorizers to integrate with existing identity providers. VPC endpoints provide fixed IPs to accommodate partner firewall requirements. Managed workflows automate post-upload processing, and AS2 connectors automate B2B file exchange. Since endpoints are charged for the time they are enabled, one approach for infrequent transfers is to compare costs with alternatives like S3 presigned URLs.
References (Official AWS Resources)
The primary sources for this page are the official AWS website and documentation. Check the official pages below for the latest specifications and pricing.
- AWS Transfer Family official page
- Amazon S3 official page
- AWS Lambda official page
- AWS Documentation (official)
If this page and the official documentation disagree, treat the official documentation as authoritative.