Well not really. I’ve literally spent the past few weeks banging my head on these things.
Especially if you want/need multi-arch. That basically requires buildx which doesn’t cache locally by default. There’s a half dozen types of caching to figure out. Then buildx is very buggy and needs qemu setup even when building natively otherwise you run into decade old bugs doing things like running sudo in a dockerfile.
It took a couple of weeks of on and off tinkering to get a stable arm builder running on a Mac m1. To get the GitHub action server to run stably and not time out was a PITA. It required IT tuning cpu limits and page caching. Not fun.
We run native machines but I would’ve much preferred a cloud solution so I could do my actual job.
I wonder how much of this is specifically Docker related pain? (I try to avoid it) It's super fascinating to see how much ARM support is coming up in this thread. I guess ARM servers are finally happening, huh. Our CI cluster has an M1 Mac in it, it took about an hour to set up. But that's doing JVM stuff with no Docker or qemu involved, so multi-arch just isn't something we need to think about and it's no different to any other machine beyond being much faster.
For servers I'd have thought you'd make a decision up front about whether to use classical x64 machines or ARM, based on load tests or cost/benefit analysis. Then you'd build one or the other. It sounds like a lot of people are putting a lot of effort into the optionality of having both, and then they are using languages and tools that can't cross-compile or JIT compile. Are you using Rust or Go or something? Hmmm.
Especially if you want/need multi-arch. That basically requires buildx which doesn’t cache locally by default. There’s a half dozen types of caching to figure out. Then buildx is very buggy and needs qemu setup even when building natively otherwise you run into decade old bugs doing things like running sudo in a dockerfile.
It took a couple of weeks of on and off tinkering to get a stable arm builder running on a Mac m1. To get the GitHub action server to run stably and not time out was a PITA. It required IT tuning cpu limits and page caching. Not fun.
We run native machines but I would’ve much preferred a cloud solution so I could do my actual job.