He also didn't say it would be "solved". He said in 2025 it would be writing almost all the code "in 12 months", but that it would also still need programmers to guide and manage it at that point. People always leave off the end of his quote.
Edit: Boris Cherny, the lead of Claude Code did say on a podcast that programming seemed "largely solved" "for the kind of programming I do" (writing harnesses I presume). Maybe that's what they were confusing it for.
The same could be lobbed against any org. "You're focused in one area, but why not solve every other problem in society simultaneously?"
But from what I can see, they do think they could fix those things you mention, because their belief in AGI as something that directly could solve healthcare issues, etc. I think their answer would be "focusing on solving AGI is the way to help the most people"
The enterprise took a wrong turn when they decided that it is the industry's goal to be the gatekeepers of intelligence, when no one could define or measure intelligence.
Some AI developers instead became the gatekeepers of something akin to "proper behavior in human language", which then morphed into "proper left-wing social/political behavior in Southern California in the early 2020s".
This is a task which calls to mind the old warning to "never wrestle with a pig: the pig enjoys it and you both get filthy".
IOW they have a scope problem. But the genii is long since out of the bottle. Effort ought to be on getting back to useful problems solving.
If they continue on their current path they risk becoming the greatest bureaucracy ever.
I don't think Pacing is talking about current models. It's talking about future models, if development continues, that may theoretically be smarter than everyone
What's an example of him crying wolf? Most of the things have either been longer term worries or short term things that have come true (eg, the worries over gpt2 were about spam and possible impersonation, which came true. His prediction that LLMs would write most code and humans would still guide them has also come true in most orgs)
There's definitely hype, but the newer models can undoubtedly do things the older ones couldn't. I had multiple long-term issues that earlier models couldn't solve (after repeated attempts) that fable did in 1 shot. On small models too, the differences in what I can trust them with has dramatically changed compared to a few months ago. Unless you breathlessly never touched the limitatioms of earlier models, it's very clear that the wall of limitations has been moving outward.
Note also when a new benchmark is released, older models do worse on it than recent models, despite none of them being trained to the benchmark.
Sure, because those businesses collect training data from users who are working on those problems. Perhaps this will never saturate and frontier users will always provide data to fill last generations gaps.
My sense is the economics of that are going to collapse. It's currently extremely expensive to be on this endless retrain and inference cycle in order just to bake in additional marginal features.
Maybe, maybe not. However I don't personally see anything other than 'one more leap', which might in any case arise from better integration with harnesses. I can foresee a step change due to harness reinforcement -- but other than that long mild refinements that are very expensive to acquire
Edit: Boris Cherny, the lead of Claude Code did say on a podcast that programming seemed "largely solved" "for the kind of programming I do" (writing harnesses I presume). Maybe that's what they were confusing it for.
reply