You are leaving out an important aspect: context switches are expensive, especially with all the mitigations introduced after Spectre et al.
Doing more work in a single thread saves a lot of context switches, especially if IO makes up a large portion of the workload. And even more so with other mechanisms to minimize syscalls, like io_uring on Linux.
Async overhead is highly dependent on the way the async runtime is implemented.
It's true that async requires scheduling and task switching to be implemented in userspace, but assuming that async is used as an alternative to threads, then the OS would be doing a similar amount of work anyway.
The OS may be able to do that more efficiently because it has more information about the system. Or it may be the opposite, because the application has more information about the workloads.
"async = overhead" as a general statement is not correct.
Thread context switches are about scheduling threads on physical cores. Async is about executing in a scope on a thread. There's no direct conversion here. Packing a thread with async tasks could let you reduce the number of threads you have, which will probably reduce the thread context switching if there is contention for cores. Whether that's significantly better than the cost of async context switches really depends on specifics (even if async context switch cost is near zero, though we're no longer talking about C++ coroutines in that case). But keep in mind: If your threads are mostly waiting, they aren't triggering a lot of context switches. More async means fewer busier threads, less async means more less busy threads. To run into a problem with less async your threads need to be forcing switching at pretty fine-grained level. (BTW, most blocking for IO isn't fine-grained from the perspective of instructions executing on a core)
That kind of thing happens, but has solutions other than async.
Do you have a citation for kernel mode having more efficient context switches? What kind of direct hardware access are you referring to that would be better than pushing the register context onto the stack?
In my experience, the exact opposite is true, particularly in the era of CPU mitigations that require TLB flushes upon every kernel-mode context switch.
You're right, kernel-level context switching is much slower than user-level context switching.
User-level can also have the advantage of having more actual context about the task that is running, meaning that it's often able to avoid saving/restoring as much data as a kernel-level switch would. See Go's green threads for a great example of this kind of cooperation between runtime and language.
> Do you have a citation for kernel mode having more efficient context switches? What kind of direct hardware access are you referring to that would be better than pushing the register context onto the stack?
The closest thing to this that I can think of is on 32-bit x86 which did have hardware assisted context switching via TSRs.
As it happens, everybody stopped using it because it was too slow, and a bit painful unless you fully bought into x86's awful segmentation model. Early Linux kernels use it if you want to see it in action.
If you have 1 thread handling 1000 requests with some async io mechanism (epoll, io_uring, ...) ,instead of 1000 threads each handling one request, there are much fewer threads fighting over CPU cores and the 1 thread can stay active much longer, hence reducing the amount of context switches.
Especially with a mechanism like io_uring, which helps minimize syscalls (and hence switching to kernel threads).
Doing more work in a single thread saves a lot of context switches, especially if IO makes up a large portion of the workload. And even more so with other mechanisms to minimize syscalls, like io_uring on Linux.
Async overhead is highly dependent on the way the async runtime is implemented.
It's true that async requires scheduling and task switching to be implemented in userspace, but assuming that async is used as an alternative to threads, then the OS would be doing a similar amount of work anyway.
The OS may be able to do that more efficiently because it has more information about the system. Or it may be the opposite, because the application has more information about the workloads.
"async = overhead" as a general statement is not correct.