I haven't seen any details from AlphaZero, but when they were training AlphaGo Zero they drastically reduced the amount of hardware and only used a single node taking about a killowatt of power. If we assume AlphaZero uses the same hardware, then given that a brain uses about 10w that would be equivalent to 800 hours. Which is still amazing.
Let me reformulate. I'm just saying that there is at least one obvious way in which Google could scale this at least 2 to 3 orders of magnitude if they would want to, for the cost of the electricity bill. Just run it for 8 months instead of 8 hours. And I wonder which problems are feasible when you look at it that way. I mean, they just solved chess in what looks like a warming-up.
>That's eight hours on thousands of parallel TPUs.
But it's not really misleading at all. The ability to produce massive amount of power and product is the pinnacle of our society. The fact is we can make thousand, if not millions or even billions of TPUs if we desired, and it is a relatively easy engineering problem at that. And that these things may solve all kinds of problems mankind has had for millennia in hours should be a wakeup call to a future that will be hard to predict.
Humans take 9 months to gestate, no amount of parallelism will speed that up, after that it takes 18 years for them not to be completely stupid all the time attempting to hammer an education in them. Even after that it takes more years to become a specialist.
44 million, according to their paper, and they used 5000 TPUs, which are capable of 4.6×10^17 operations per second.
(The operations the TPU can run are far simpler than what supercomputers can do, but just for the sake of comparison, the current top supercomputer in the world can do 1.25×10^17 floating point operations per second)
