Yes, sorry, I'm referencing the down-clocking, which has an impact on other instructions that run. What Cloudflare were running in to was that AVX512 optimisations were only available on certain ciphers in OpenSSL, and weren't making up the majority of calls. They made up enough to be continually punting the chips down in to a lower core speed and significantly impacted other non-AVX instructions running on the system. Cloudflare ran in to this in part because they were opting for cheaper Silver class chips that are more likely to down-clock than Gold or Platinum. Bronze is worse.
Right, downclocking is a different matter and I'm aware of the pitfalls there. As you point out, the extent of the downclocking varies based on the model - and also the exact "license" chosen depends not only on the instructions used, but their relative intensity (e.g., you can do a lot of 512-bit FMA instruction without suffering the slowest speeds, but once FMAs are dense enough you'll get the downclocking).
FWIW, I recently tested a consumer Cannonlake chip and there is no downclocking even with heavy AVX-512: all types of tested code ran at the full turbo frequency. So it seems that not all chips will have these varying frequencies based on ISA extensions used.
Oh wow. That makes this even more of a mess than I'd realised. This seems an optimisation nightmare. You'd need to know a lot about the runtime and running environment to be able to figure out whether or not to enable AVX-512 instructions. Seems like something only JIT runtimes would be able to realistically handle on the fly.
Contrast:
Bronze: https://en.wikichip.org/wiki/intel/xeon_bronze/3106#Frequenc...
Silver: https://en.wikichip.org/wiki/intel/xeon_silver/4108#Frequenc...
Gold: https://en.wikichip.org/wiki/intel/xeon_gold/5118#Frequencie...
Platinum: https://en.wikichip.org/wiki/intel/xeon_platinum/8153#Freque...