You can run an RTX 3080 off anything with enough PCI bandwidth to handle it. Presumably the same goes for Apple's GPU. We could adjust for CPU wattage, but at-load it amounts to +/-40w on either side and when we're only testing the GPU it's like +/-10w maximum.
The larger point is that Apple's lead doesn't extrapolate very far here, even with a generous comparison to a last-gen GPU. It will be great at inferencing, but so are most machines with AVX2 and 8 gigs of DRAM. If you're convinced Apple hardware is the apex of inferencing performance, you should Runpod a 40-series card and prove yourself wrong real quick. It's less than $1 and well worth the reality check.
My point was mostly that the 200W TDP you quote is for the whole package (CPU, GPU, RAM, plus the Neural network thingy and the whole IO stuff). A 120W figure for the GPU is more realistic.
I'm not pretending the Apple chips are the be-all-end-all of performance. They certainly have limitations and are not able to compete with proper high end chips. However I can confidently say that on mobile devices and laptops, competition is largely behind. Sure a 1000+$ standalone GPU will be faster, but it doesn't fit in my jeans. It's the same as comparing a Hasselblad camera with the iPhone 14 pro...
The competition is all fine, though. They have enough memory to run the models, they have hardware acceleration (ARMnn, SNPE, etc.) and both OSes can run it fine. Apple's difference is... their own set of APIs and hardware options?
How can you justify your claim that they're "largely behind"? It sounds to me like the competition is neck-and-neck in the consumer market, and blowing them out at-scale. It's simply hard to entertain that argument for a platform without CUDA, much less the performance crown or performance-per-watt crown.
The larger point is that Apple's lead doesn't extrapolate very far here, even with a generous comparison to a last-gen GPU. It will be great at inferencing, but so are most machines with AVX2 and 8 gigs of DRAM. If you're convinced Apple hardware is the apex of inferencing performance, you should Runpod a 40-series card and prove yourself wrong real quick. It's less than $1 and well worth the reality check.