Simple Cloud Infrastructure for Small Teams
A technical deep dive into running a Ruby on Rails application on Hetzner Cloud using Terraform, Ansible, and Docker Swarm. The episode highlights how small teams can reduce costs by 70% while maintaining reliability through boring technology and single-node database strategies.
Strategic Shift to Boring Technology
Small engineering teams often over-engineer their infrastructure, adopting complex tools that increase operational debt. This analysis highlights a successful case study where a six-person team migrated from AWS to Hetzner Cloud, achieving a 70-75% reduction in infrastructure costs while maintaining reliability. The core strategy relies on "boring technology": using stable, well-understood tools like Terraform, Ansible, and Docker Swarm Mode rather than cutting-edge, high-maintenance solutions.
Infrastructure Architecture
The team utilizes a simple cloud provider to avoid vendor lock-in, ensuring that no specific feature is exclusive to one provider. Infrastructure is managed entirely through Terraform, providing version-controlled, executable documentation. Ansible handles machine provisioning, ensuring that new instances are self-sufficient and can boot without manual intervention. This setup allows for rapid scaling and easy recovery from hardware failures, a common occurrence in cloud environments.
Database and Data Strategy
A controversial but effective decision is the use of a single-node Postgres database without a hot standby. The team argues that for many business applications, the cost of high availability is not justified by the risk of downtime. Instead, they rely on PG Backrest for Point-in-Time Recovery, storing backups in S3-compatible storage. This approach simplifies operations and reduces costs, with maintenance windows scheduled during low-traffic hours to minimize user impact.
Deployment and Operations
Deployments are streamlined using Capistrano to execute Docker Stack updates. The use of Docker Swarm Mode provides zero-downtime deployments by ensuring new containers are healthy before traffic is routed. This eliminates the need for Kubernetes, which adds unnecessary complexity for small teams. The team emphasizes that their infrastructure code has had minimal changes over five months, indicating a stable and reliable setup.
Conclusion
For small teams, the priority should be on reducing operational overhead and costs. By choosing simple, portable tools and accepting manageable risks in database availability, businesses can achieve significant cost savings without sacrificing reliability. This model is particularly effective for teams that are fully remote and need to communicate asynchronously, as the infrastructure is self-documenting and easy to understand.
Key insights
-
Small teams benefit significantly from adopting boring, stable technology over complex, modern solutions. This reduces the cognitive load on developers and minimizes unexpected operational issues.
Impact: Increases developer productivity and reduces the risk of infrastructure-related outages, allowing the team to focus on core product features.
-
A single-node database with robust Point-in-Time Recovery is a viable alternative to high-availability clusters for many business applications. The cost savings often outweigh the risk of brief downtime.
Impact: Reduces infrastructure costs by eliminating redundant database instances and simplifies backup and recovery procedures.
-
Infrastructure as Code (IaC) using Terraform is essential for avoiding vendor lock-in. It allows for easy migration between cloud providers and ensures that infrastructure is reproducible and version-controlled.
Impact: Enhances business continuity by enabling rapid migration to alternative providers if costs or service levels become unfavorable.
-
Docker Swarm Mode is a sufficient orchestration tool for small to medium-sized applications. It provides zero-downtime deployments without the complexity of Kubernetes.
Impact: Simplifies the deployment process and reduces the need for specialized DevOps expertise, making it accessible to smaller engineering teams.
-
Monolithic architectures are more efficient for small teams than microservices. They simplify deployment, debugging, and maintenance, reducing the overhead of managing multiple services.
Impact: Improves development velocity and reduces the complexity of the codebase, leading to faster feature delivery and easier maintenance.
Action items
-
Audit current infrastructure for unnecessary complexity and replace it with stable, well-documented tools. Focus on reducing the number of moving parts in the deployment pipeline.
Impact: Decreases operational overhead and improves system reliability by minimizing the surface area for potential failures.
-
Evaluate the necessity of high-availability database clusters. Consider implementing a single-node setup with Point-in-Time Recovery if the business can tolerate brief maintenance windows.
Impact: Significantly reduces database infrastructure costs while maintaining data integrity through robust backup strategies.
-
Implement Infrastructure as Code using Terraform for all cloud resources. Ensure that all infrastructure changes are version-controlled and can be reproduced in staging environments.
Impact: Prevents vendor lock-in and ensures that infrastructure is consistent across environments, reducing configuration drift and deployment errors.
-
Adopt Docker Swarm Mode for zero-downtime deployments if Kubernetes is not required. Configure health checks to ensure new containers are ready before traffic is routed to them.
Impact: Simplifies the deployment process and reduces the need for complex orchestration tools, making it easier for small teams to manage production environments.
-
Review the application architecture and consider consolidating microservices into a monolith if the team size is small. This reduces the complexity of deployment and maintenance.
Impact: Improves development efficiency and reduces the operational burden of managing multiple services, allowing the team to focus on core business logic.
Quotes
“Das ist so ein bisschen das, was wir euch erzählen wollen. Und darum haben wir überlegt, wir wollen euch da so ein bisschen durchführen.”
“Wir haben uns explizit dafür entschieden, einen Monolithen zu bauen. Die Anwendung ist ein paar Mal zwischendurch neu geschrieben worden.”
“Wir haben auf jeden Fall nicht bereut, so ein einfaches Setup zu haben von einem Single-Node mit ordentlichem Point-in-Time-Recovery.”