Hello everyone. I was recently hired at an established company that is in the process of migrating to AWS from Heroku. I am the second devops/SRE like employee, the other person has more of an IT background. He, along with the 20ish person dev team, have no experience with anything cloud or IaC. Our workloads are almost entirely stateless web services, with some data pipeline work that has already been moved to AWS lambda.
My gut is telling me ECS is the better choice for this team - simpler, no operational maintenance, almost no additional abstractions beyond the AWS primitives people will need to learn anyway. A consulting team hired before I was brought in are pushing EKS, but I’m not convinced that the improved devex is worth the increase in complexity and operational burden for our team. Their argument is that developers can be effectively quarantined within k8s, so the increase in maintenance on the cluster itself is offset by the reduction in work around custom IAM permissioning, multiple AWS accounts (we’d just use namespaces within the cluster), etc.
Any thoughts or input would be greatly appreciated.
ECS Fargate. We used it for a lot of stateless containers, similar as you mention.
I tried to migrate from ECS Fargate onto EKS, but moved back after a while. The reason was the "management" overhead of babysitting the cluster and it's non-app deployment like cilium, fluxcd etc. - Since those were part needed and part QoL features we came to the conclusion that ECS Fargate was the better and less 'head ache' solution. I would do it again. Especially not thinking about provision new nodes etc.
Sure you pay for that comfortable solution, but it was worth in my opinion
Go with ECS if you don’t need Helm and significant operational overhead. Having used both ECS and EKS for the past few years, I’ve found that ECS requires negligible maintenance.
BTW: Checkout ecs fargate too. if workload is small this will do the most.
If you don't need anything beyond the feature set of ECS and are happy to stick with AWS for the foreseeable future, then I'd advise you to listen to your gut - EKS is more complex, and needs a bit more looking after.
Personally, I'd disagree with your consultants on being able to avoid multiple AWS accounts as a selling point. When you need to isolate workloads or implement hard security boundaries, accounts are generally the simplest and most reliable way of doing that. Following the Well Architected Framework is a good standard practice and there are plenty of guides and examples to help you.
If things do change in the future, it's not actually that big a deal to deploy to EKS instead (or as well) - but for your needs, I think ECS would make a better starting point.
(And I agree with other posters that ECS Fargate is likely the best fit for you)
And you don’t know the big picture, but the little picture is everyone agreed the consultants were the right way to decide before you were there; arguing implies the people who sign contracts and write the checks are wrong; and business decisions are about the whole business and engineering exists to support the business - not to optimize itself.
To put it simply, the decision has been made by people with more experience in your organization’s culture, more authority, and a better understanding of the business as a whole.
Plus your here on HN because you have not convinced the stakeholders.
That really depends on they type of consultancy, the terms of the engagement, and where the responsibility lies (for both the requirements and the outcomes).
If the consultancy was brought in to do the move from Heroku but not build or run anything beyond that, then I would normally expect the client to set the tradeoff between DevEx (argues for EKS) vs simplicity/operability (argues for ECS). In that case, it would be perfectly reasonable for the OP to input into that decision.
It's definitely worth speaking to whoever is responsible for engaging the consultancy - there should be an agreed Statement of Work and at least a rough RACI matrix for each deliverable, showing who the stakeholders are and how they interact.
Have done both and I'd say ECS wins almost always, as long as you can be in AWS. If you are true unicorn/hyperscale, Kube will start to pay off. But otherwise, the upgrade treadmill of kube and the blank stare of SWE's who don't want to learn helm charts, argo, ingress controllers etc are not worth dealing with when you can just ship ECS and have it work for years with minimal overhead.
startups/small-medium businesses implementing Kube are doing it for cargo cult reasons or stealth career development/employment security initiatives by the relevant devops or sre teams.
ECS fargate is a must as well because no "server" means a ton less compliance burden.
My gut is telling me ECS is the better choice for this team - simpler, no operational maintenance, almost no additional abstractions beyond the AWS primitives people will need to learn anyway. A consulting team hired before I was brought in are pushing EKS, but I’m not convinced that the improved devex is worth the increase in complexity and operational burden for our team. Their argument is that developers can be effectively quarantined within k8s, so the increase in maintenance on the cluster itself is offset by the reduction in work around custom IAM permissioning, multiple AWS accounts (we’d just use namespaces within the cluster), etc.
Any thoughts or input would be greatly appreciated.
ECS Fargate. We used it for a lot of stateless containers, similar as you mention. I tried to migrate from ECS Fargate onto EKS, but moved back after a while. The reason was the "management" overhead of babysitting the cluster and it's non-app deployment like cilium, fluxcd etc. - Since those were part needed and part QoL features we came to the conclusion that ECS Fargate was the better and less 'head ache' solution. I would do it again. Especially not thinking about provision new nodes etc.
Sure you pay for that comfortable solution, but it was worth in my opinion
Go with ECS if you don’t need Helm and significant operational overhead. Having used both ECS and EKS for the past few years, I’ve found that ECS requires negligible maintenance.
BTW: Checkout ecs fargate too. if workload is small this will do the most.
If you don't need anything beyond the feature set of ECS and are happy to stick with AWS for the foreseeable future, then I'd advise you to listen to your gut - EKS is more complex, and needs a bit more looking after.
Personally, I'd disagree with your consultants on being able to avoid multiple AWS accounts as a selling point. When you need to isolate workloads or implement hard security boundaries, accounts are generally the simplest and most reliable way of doing that. Following the Well Architected Framework is a good standard practice and there are plenty of guides and examples to help you.
If things do change in the future, it's not actually that big a deal to deploy to EKS instead (or as well) - but for your needs, I think ECS would make a better starting point.
(And I agree with other posters that ECS Fargate is likely the best fit for you)
Don’t argue with the consultants.
It is not a hill worth dying on.
And you don’t know the big picture, but the little picture is everyone agreed the consultants were the right way to decide before you were there; arguing implies the people who sign contracts and write the checks are wrong; and business decisions are about the whole business and engineering exists to support the business - not to optimize itself.
To put it simply, the decision has been made by people with more experience in your organization’s culture, more authority, and a better understanding of the business as a whole.
Plus your here on HN because you have not convinced the stakeholders.
Good luck.
That really depends on they type of consultancy, the terms of the engagement, and where the responsibility lies (for both the requirements and the outcomes).
If the consultancy was brought in to do the move from Heroku but not build or run anything beyond that, then I would normally expect the client to set the tradeoff between DevEx (argues for EKS) vs simplicity/operability (argues for ECS). In that case, it would be perfectly reasonable for the OP to input into that decision.
It's definitely worth speaking to whoever is responsible for engaging the consultancy - there should be an agreed Statement of Work and at least a rough RACI matrix for each deliverable, showing who the stakeholders are and how they interact.
Have done both and I'd say ECS wins almost always, as long as you can be in AWS. If you are true unicorn/hyperscale, Kube will start to pay off. But otherwise, the upgrade treadmill of kube and the blank stare of SWE's who don't want to learn helm charts, argo, ingress controllers etc are not worth dealing with when you can just ship ECS and have it work for years with minimal overhead.
startups/small-medium businesses implementing Kube are doing it for cargo cult reasons or stealth career development/employment security initiatives by the relevant devops or sre teams.
ECS fargate is a must as well because no "server" means a ton less compliance burden.