Building a Managed SFTP Server with AWS Transfer Family - S3 Integration and User Management
Learn how to build SFTP/FTPS servers with Transfer Family, integrate with S3, and configure custom authentication.
Overview of Transfer Family
Transfer Family is a service that provides managed servers supporting four protocols - SFTP, FTPS, FTP, and AS2 - capable of handling up to thousands of concurrent connections. If you use SFTP for file exchange with trading partners, you can build an SFTP server with Transfer Family and set S3 as the transfer destination, freeing you from file storage management. Existing SFTP clients and scripts can be migrated simply by changing the connection hostname. EFS (Elastic File System) can also be selected as the backend storage for workloads requiring POSIX permissions.
Authentication and Workflows
User authentication can be configured using one of three methods: service-managed (managed within Transfer Family), custom (Lambda authorizer), or AWS Directory Service. Custom authentication lets you implement authentication logic in a Lambda function and integrate with existing LDAP or Active Directory. The Lambda authorizer response can dynamically determine per-user home directories, IAM roles, and S3 bucket prefixes, supporting multi-tenant configurations. Managed workflows are processing chains that run automatically after file uploads, allowing you to define file copying, tagging, PGP decryption, and custom Lambda processing. Workflow steps can include conditional branching (file name patterns, size) to execute different processing based on file type.
AS2 Protocol and Automation
The AS2 (Applicability Statement 2) protocol is widely used for B2B file exchange, providing message encryption, signing, and delivery receipts (MDN). Transfer Family's AS2 connector lets you set up automated file exchange with partners using S3 as storage. Managed workflows execute automated processing (copying, tagging, Lambda invocation) after file uploads, building file processing pipelines. CloudWatch Logs records transfer logs for auditing file send and receive activity. For detailed information about Transfer Family, you can also find related books on Amazon.
Design Best Practices and Pitfalls
Endpoint type selection is the first critical design decision. Public endpoints are simplest to configure but do not provide fixed IP addresses, making them incompatible with partner firewall allowlists. VPC endpoints allow fixed IP assignment via Elastic IP, enabling IP notification to partners. VPC-internal endpoints restrict access to VPN/Direct Connect connections only, completely blocking internet access. A common pitfall is overlooking the always-on endpoint cost (approximately $216/month). For small-scale use cases with transfers once daily, the cost-effectiveness is poor and comparison with S3 presigned URLs or DataSync on-demand transfers is recommended. Another frequent issue is conflicts between S3 bucket policies and user IAM role permissions causing unexpected access denials. Grant users' IAM roles minimum permissions to the S3 bucket only, and explicitly allow access from the Transfer Family service role in the bucket policy.
Comparison with Alternative Services
File transfer methods compared with Transfer Family include S3 presigned URLs, DataSync, and Storage Gateway. S3 presigned URLs require no always-on server and have the lowest cost, but they require partners to change their SFTP clients (migrating to URL-based transfers), breaking existing script compatibility. DataSync specializes in bulk data migration from on-premises with excellent bandwidth control and scheduling, but does not support inbound transfers (push-type) from partners. Storage Gateway (File Gateway) provides NFS/SMB mount points on-premises but is not suited for exposing to external partners. Transfer Family's strength is summarized as 'migrating to the cloud without changing existing SFTP workflows at all.' When the political cost of requesting changes from partners' IT departments is high, Transfer Family's endpoint maintenance cost is justified as migration friction avoidance. FTP (unencrypted) should not be adopted for new deployments as it fails security requirements - use SFTP or FTPS instead.
Transfer Family Pricing
Transfer Family pricing consists of protocol endpoint hours (approximately $0.30 per hour, approximately $216 per month) and data transfer volume (approximately $0.04 per GB for uploads, approximately $0.04 per GB for downloads). Since the always-on endpoint cost is the primary expense, evaluate cost efficiency in advance if transfer frequency is low. When serving multiple protocols (SFTP + FTPS) on the same server, only one server's endpoint fee applies. S3 storage costs are billed separately. Managed workflow step execution incurs additional charges, so minimize the number of steps when processing large volumes of files. AS2 connectors are billed separately by connector hours, requiring cost estimation based on partner exchange frequency.
Summary
Transfer Family provides managed servers for SFTP, FTPS, FTP, and AS2 protocols, simplifying file storage management through S3 integration. Custom authentication uses Lambda authorizers to integrate with existing identity providers. VPC endpoints provide fixed IPs to accommodate partner firewall requirements. Managed workflows automate post-upload processing, and AS2 connectors automate B2B file exchange. With always-on endpoint costs of approximately $216/month, comparing alternative approaches like S3 presigned URLs is important when transfer frequency is low.