Does anybody actually know whether there's a limit to the complexity LLMs are capable of dealing with in a codebase? It's very obvious that they don't write code that is suitable for people to understand it (and it's gonna get worse and worse the more RL is used to train these models), but if there isn't a point at which LLMs also struggle due to the complexity they introduce, then I'm not sure it really matters anymore for a large part of non safety-critical software. I really hope there is, because steering them is, I feel, one of the last competencies through which I can still add value, but is there actually evidence that these models struggle more with poorly maintained code?
> I really hope there is, because steering them is, I feel, one of the last competencies through which I can still add value
Something as simple as output length is a hard linear floor for productivity, even putting aside the obvious context problems that you're intuiting, and it's far from being the biggest cost that arises from steering skill. Learning to make a smaller, faster model do the same work with less tokens is a technical domain that a lot of people don't seem capable of learning. I'm not just talking about "context engineering", but learning how to fine tune, post-train, create better harnesses, design inference setups, etc. If we're both using AI, but I'm beating you to market every single time and with a better product, what is your AI usage actually buying you? Yes, competency and skill is this meaningful right now, and it's highly technical. Not the least of which because you know how to describe the problem in way that gives it a smaller solution and requires less iteration.
Most of the labor who understand the technology enough to do those things lives at the companies selling you these services, but you can absolutely learn to do these things yourself right now. It's actually really fun! A hell of a lot more fun than fucking prompting that's for sure.
Where we're at, I would equate it to the early mainframe era where the programmers came with the computer. I'm placing calls that we follow a similar track and the two will end up decoupling, that "model engineers" are going to move in-house. OpenAI will have a ring to it like IBM does today.
Yeah, exactly my experience. Especially ever since Fable came out, I felt paralysed, many times, about how many decisions I have to hand it. Which is 10x harder when you handed it the reins to build the PR and you practically only knew the compressed initial problem statement. Then it stands up a draft PR and says: "Decisions you owe me: [insert bulleted list with 8 items each a paragraph long]" and you are like: "...oh shit".
It becomes a very heavy and difficult exercise of it walking you through the implementation and the judgment calls it had to make along the way. VERY exhausting.
There's a silver lining though: you do get to gradually clarify a proto project spec and various requirements, but boy does it take time and energy to re-contextualise when the bot tells you that you should make decisions.
Well, of course Anthropic employees would say that, since they likely do the same. Claiming that your primary competitor doesn't engage in a certain malicious practice is supposed to make it look as if there's no way you would too. If somebody even says that about their competitor, then surely there must be truth to that, otherwise you would never give credit to someone you're opposed to.
Is it really interesting though? It's essentially just vibe-coded by people who are unqualified for this kind of work. One of the authors claimed that what qualified them was having worked on a large-scale postgres cluster; they never actually worked on databases or compilers.
Have you considered the possibility that you're simply not as good as the experts, and that your experience of LLMs being capable of performing your work up to your standards doesn't imply that experts are necessarily in denial?
Being fairly at expert level in a "solved" domain (for some definition of solved) but also having near-expert level proficiency in a non-technical as-yet "unsolved" domain and watching the process repeat there (and watching how people react as inroads are made progressively deeper) is basically my own standpoint. But I am intentionally using "solved" and "unsolved" very loosely here: you can get stuck in the thicket of arguments about verifiable domains, what it means for a domain to be solved, whether a set of evals can tell us something has been solved or not, and so on. One way to avoid that (as my original comment pointed to) and focus on what matters for us is to look instead at the effects being produced in the work process and on the division of labor as a whole.
I desperately hope that the Andrew Kelley style of software engineering will survive all of this; that users will continue to value quality and not be content with slop. This, of course, presumes that products built fully by agents will produce sub-par quality in the future. If they will be able to manage to glue all of this slop together without the project collapsing in on itself, none of this will matter. I just hope that this isn't the future of the industry.
Most of these republicans/libertarians only want the government to leave them alone. They don't care when a company they aren't affiliated with is regulated. You can see Marc Andreesen celebrating the government's decision on Anthropic. Similarly, when Silicon Valley Bank went bankrupt, libertarians such as David Sacks were loudly calling for government bailouts. It's just hypocrisy all the way up.
The entire movement of conservatism in America is a propaganda operation oriented around manufacturing consent for a return to the Gilded Age. It is entirely bankrupt of morals and has been from the beginning. If you personally are a conservative, now is a good time to take a good hard honest look at the history of your movement in American politics. There might even still be time to realign yourself with a movement that isn't actively seeking to harm you.
This is garbage reposted in every HN article that starts to talk about any related to politics.
I will gladly live in a conservative county over progressive one. And reading this paragraph in a article about AI is complete nonsense. This is a go touch grass moment if you needed one.
I personally repost it everywhere because it is an hypothesis that I believe has strong weight of evidence behind it, and I think it's important to repeat.
Your personal preferences and beliefs have little to do with conservatism at large and the motivations of the powerful people who promote it. If you want to reach a place of open minded debate and discussion in which there can be different legitimate approaches to governing, you have to start with an honest assessment of the world as it is, not as you would like it to be.
The reason it's relevant in an article about AI should be self-evident. AI is powerful, the industry is already massive, and the leaders in that industry are involved in quite a bit of political maneuvering. You may choose to ignore politics, but politics will not ignore you.
The Democratic Party are the one losing elections they should trivially have won therefore it is clearly the Republicans, vile as they are, that have a more "honest assessment of the world as it is".
How is that relevant? I never mentioned any political party. But now that you mention it, look up the Southern Strategy, a lot of this stuff dates all the way back to Goldwater.
"The entire movement of conservatism in America is a propaganda operation"
This is nonsense. I could easily say the same thing about progressives highlighting/cherrypicking some of the worst living situations in America today.
The label of libertarian is thought of as a binary by non-libertarians, leading to this perception of hypocrisy, but that is not the way actual libertarians think with the exception of a tiny minority. Libertarianism is a spectrum, just as any other political affiliation or belief system.
This idea that if some group isn’t all reading from the same sheet of music you imagine they should be reading from means they are hypocrites is just wrong.
Another potential reason, not mentioned in the article, is that open source models obviously pose the biggest threat in the labs' ability to monetize their tech. Anthropic especially seems to be very anti open-source. If frontier models start to plateau and don't have capabilities that truly differentiate them, nobody will pay what the labs would want to charge. Posing the tech as a danger is a way for them to make the government regulate open source models.
This is a great point. I'm kind of surprised there isn't a greater proliferation of open source models to do things the public ones won't. I know such things exist, but imagine how many web browsers there would be if all the mainstream ones had the same content restrictions as LLMs.
I guess since training them does take cash that raises the bar for what people will do as a prank or on principle.
Training is time-consuming and/or expensive but it is not the main blocker.
The main problem is obtaining a big enough training data set. Now, unless you are someone like Google or Microsoft, it has become much harder to scrap data from the Internet than by the time when OpenAI and Anthropic got most of their data.
To me it makes absolutely zero sense that they would decide to not release the model to the public because of the effects that it would have due to its exploitation capabilities. Previous models were also capable of providing harmful information, yet that wasn't a problem, because models can actually be effectively censored using RHLF. So what is preventing Anthropic to simply forbid the model from letting people vibe-code exploits???
Fully agree here. I think it’s evident more that mythos does not deliver step change results and more is a disappointment.. so let’s hype it up by further scaring the masses of its ‘mythic’ abilities
reply