Id say that it is a lot more likely for the in-house, at least half a decade old library to be correct and performant than 3000 lines of an LLMs mediocre regurgitation of that code.
I think software engineering in general is in a bit of a discoverability crisis. So many problems actually have solutions implemented... Somewhere. If you know about them. And are speaking the same vocabulary as the original implementer to realize the solution might be applicable to your problem. It's one of the reasons that jokes exist about microservice frameworks (https://www.youtube.com/watch?v=y8OnoxKotPQ) and how "We use Hadoop to store the output from our Kafka pipe, that's populated from our Traefik layer, all monitored with Grafana in front of Loki and Prometheus, of course" is a real sentence that has actual meaning and not a fever-dream.
LLMs are actually pretty impressive at being able to pull together disparate information from various domains into one place.
I'm not sure I follow. C++ just added a SIMD library for this exact problem. And even then, SIMD-intrinsic-free C++ with the right data structures and some basic hardware understanding gets you mostly there in a hardware agnostic portable way.
They've added a new library to a language and stdlib so crammed full of features that its specification exceeds the size of the King James Bible by wordcount.
That's what I mean about "discoverability crisis." This post is how I learned about the existence of the SIMD library.
I'd absolutely still use Highway, and do. My experience is that even two separate implementations diverge over time and I'd have low confidence in bringing the same updates and improvements to all, even with LLM assistance.
Our programming model is 1) an agent+human to generate the algorithmic approach, 2) a C++ library (Highway) to translate to intrinsics, while filling in gaps + allowing customization, 3) a compiler to generate the actual code with some optimizations.
Asking the compiler to do #1 is a pipe dream: compiler friends tell me they are not going to devise new shuffles/data layouts (like what VQSort does). Conflating #2 and #3 means a custom compiler/IR which has high engineering costs (ABI boundaries, hard to debug/profile/sanitize).
And doing #3 at runtime (JIT), or moving fusions into #3 (MLIR), vastly complicates the compiler. We can still get runtime adaptability thanks to Highway's multi-target support. Fusion has been much easier to implement manually for LLMs than to construct a general fusion infrastructure. Templates hide most data type differences and we see 2-5x speedup vs llama.cpp for 128k prefill+batch decode on Zen5.
Instead of requiring a compiler to do heroic transforms at runtime, and get it right every time, we can do all kinds of agentic exploration, then verify the result/approach, check in the source code, then we 'just' have a C++ compiler afterwards. And if/when something breaks, it's easy to update centrally, in code we can modify directly, rather than indirectly via updating a compiler.
Nothing absurd about that. What do you think an "Executive producer" is? A "Director" ? Does Peter Jackson get credit for creating the Lord of the Rings Trilogy films? Christopher Nolan for his films? But did he make them ? No, it was the collective effort of thousands of individuals all working under their direction.
Just like if somebody creates software today, and the end result is generated by the collective effort of thousands of agents, the "Director" still gets credit.
What does it matter who receives more credit? The producer produces the film and the director directs it. If I pick my phone and record a video, I'm now the producer and director of the film and the sole creator of it.
If there were multiple people involved in the creation of a film I helped to create, I cannot factually say I created it. Just like if someone builds something using code generated by AI, they can't factually say they created it.
There's a bit more nuance to it though, I think, because traditionally there has been more of a firm line between humans and tools.
For example, most people would agree with this line you wrote: "If I pick my phone and record a video, I'm now the producer and director of the film and the sole creator of it." A pedant could say "woah hang on, that's ignoring the fact that actually the iPhone is the one recording the images that make up the film, how can the human get all the credit", yet nobody would actually make that argument when discussing who created the film.
Generative AI is pretty much the first time (maybe there are some niche contradictions to this claim?) we consider a tool to be contributing enough creativity to the process that we don't all agree "only one person was operating this tool, so that person is the sole creator" - but some people DO still hold that line, and do consider the human who wrote the prompt to be the creator.
And I don't think there's any objective technical metric we can use to say who's right, it comes down to our collective judgement deciding where the line is.
That’s an interesting way to look at it, and still generous to the llm.
By which I mean it puts the llm in the role of actor and such… people practicing an art. It seems harder to make a case that the llm is actually making creative decisions rather than aping a synthesis of past human decisions and expressions of taste, strictly, at every turn.
Though I guess it’s all “shades of grey”. I never felt like I was contributing a ton when I was essentially wrapping a large, capable library that some smarter person or people made, either.
Tangentially… remember back when we used to see headlines like “12 year old made a web browser!” and we all thought, “No. No they didn’t.”
Its a nightmare because it erodes trust. Doctors are not "always right" which is why "always get a second opinion" is codified in culture.
But AI's problem is that its completely full of shit, sometimes, and the people most qualified to evaluate whether its full of shit are the doctors, not the patients, but just like OP's original article, patients are left feeling like their second opinion from AI might be more trustworthy than their doctors opinion.
> But AI's problem is that its completely full of shit, sometimes
It's now quite unusual that it's "Completely full of shit". If it contradicts something your doctor said I don't see why you should feel ashamed to bring it up.
Sure it complicates the doctor's work, having ignorant obedient patients must be more comfortable for the doctor, but the end result could be more accurate diagnosis.
But those are obvious errors. What if the AI tells you to up your intake of X and it seems plausible? So you up your intake of X by taking some supplement, but upping the intake of X makes your body deplete more of Y and now you have a new or compounding problem.
A doctor might have never recommended upping X, because they would know what it does to your body. Or they might have suggested additional supplementation to avoid this.
The fact that LLMs are trained on all public knowledge is a huge red flag, because there are more wrong infos out there than right ones. Especially about health, diet, etc.
I was about to respond but your last statement implies fundamental misunderstanding of how LLMs work. I don’t think you even know about RLHF and you think good and bad ideas are spread in proportion to how much they are seen on the internet.
I think my favorite part would be where they were unbolting entire seats and feeding them directly into industrial shredders.
"Ford imported all of its first-generation Ford Transit Connect models as "passenger vehicles" by including rear windows, rear seats, and rear seat belts.[1] The vehicles were exported from Turkey on ships owned by Wallenius Wilhelmsen Logistics (WWL), arrived in Baltimore, and were converted back into light trucks at WWL's Vehicle Services Americas, Inc. facility by replacing rear windows with metal panels and removing the rear seats and seat belts.[1] The removed parts were not shipped back to Turkey for reuse, but shredded and recycled in Ohio.[1] The process exploited the loophole in the customs definition of a light truck; as cargo does not need seats with seat belts or rear windows, presence of those items automatically qualified the vehicle as a "passenger vehicle" and exempted the vehicle from "light truck" status. The process cost Ford hundreds of dollars per van, but saved thousands in taxes.[1]"
"killed" is a bit of a stretch. High prices on all gear is here to stay. This is the new normal. Unless that simply means that nobody buys consoles/pc's.
But you cant compare the price point with what it used to cost and imagine that its overpriced now and that people will seek alternatives. There aren't any cheaper alternatives.
It almost certainly is. Once people get used to the higher prices, and the companies see that the units sell anyway, there is no meaningful incentive to lower costs again.
This has played out time and time again during every other supply-side shock. Once prices go up, they don't come back down.
That's not true. We've seen prices from supply shock go back down (the increase in hard drive prices when there were floods in Asia 10-15 years ago comes to mind as an example). It does take a while, but it will happen eventually.
Even then, prices stayed elevated for years. They never went back to pre-supply shock prices. Right around that time too the industry consolidated. WD bought Hitachi, Seagate bought Samsung's HDD business. It left a duopoly, so now there was no competitive pressure for a price war. Prices got locked in higher than pre-flood levels, intentionally.
For the current DRAM situation, I can almost promise we'll never see $60-$90 RAM again. Maybe, 32GB won't cost you $500 eventually, but it'll cost you $250-$350 instead of $500. If the market can bear it, why would anyone get into a price war that's just a race to the bottom where no one wins?
For the eggs I buy, they are currently $7.99/dozen. Cheapest in my store is $5.99/dozen.
That doesn't disprove my point though. Prices are still higher as a baseline than before the supply side shock. Prices raise to a "new normal" and consumers adapt, removing pressure to lower back down to pre-shock levels.
wholesale egg prices have actually plummeted, yet retail prices have only drifted slowly downward incrementally, and have not reached the previous baseline. Its asymmetric price transmission, and its a documented economic phenomenon. "Prices go up like rockets, and fall like feathers"
Eggs in USA literally hit a ten year low a few months ago. I don't know where you live, probably SF or something, but a dozen eggs here is under 3 dollars.
PC components market has historically been highly competitive so I’m pretty sure it wouldn’t quite work out for vendors who tried doing that (aside from GPUs/Nvidia)
It's also because television producers found alternative revenue streams that allow them to sell the TVs for less while still making more. If you look for a TV without all of the adware/bloatware/spyware you can see the true cost of a TV in 2026.
You have evidence that they are going to go down? Not unless government policy steps in to pressure chip makers, or establish new markets. Corporations will use inflation, ai, et al to validate their record profits at the cost of the consumer. Monopolies or better put the mergers of companies over the last 40 years hasn’t lead to cheaper prices, it never was going to either.
Can you point to an example of this happening in the past? Where a supply shortage leads to price increases and "record profits", and the price never goes back down?
Good point. But only a few companies create these things. They can jack up the price and there is nothing we can do. Is there a mom and pop shop making memory yet? Nope, centralized power of commerce is a threat.
I think the point is that "only a few companies create these things" has always been true.
You need to provide a compelling argument why it is different this time.
The counter-argument is pretty basic:
RAM companies are currently selling as much as they can at high prices. This leads to investment in building new factories.
At some point the supply of new RAM will match the demand for it. When that occurs companies can increase profit by cutting prices to gain market share.
What's more, all the RAM companies have slightly different estimates of what the demand is. This leads to different levels of investment in new factories. Some will over-invest in new factories and the only way they can make their investment back is by increasing market share.
The final factor is new entries in the market. Chinese RAM manufactures can already produce DDR4 RAM (but only small amounts of DDR5). They can both increase supply of DDR4 RAM and are aggressively chasing DDR5 capabilities.
TL;DR: The profit motive is too strong for companies to artificially keep prices high once demand drops.
There's a lot of money behind AI to try to make fetch happen but every attempt to capture the real cost of running these models has driven home to people that we're still deep in the "burn money to acquire customers" phase before the "start charging people gobs of money to make a profit" turn. All the stories of companies burning through their whole year of AI budget in the recent move from subscription to usage based billing is a big example.
If that bubble pops like it seems to be threatening to do memory prices could drop back to their old levels give or take some sticky inflation.
Suppose you have a warehouse full of widgets. You bought them them for $450 each, and sell them for $500. You're really happy with this profit, and you can just keep selling them at $500...forever, right?
But then, I get my own warehouse and fill it with widgets that I bought at $400 each because I entered under better market conditions. And I really want to sell these widgets -- they aren't making me any money when they just sit there taking up space and burning rent.
So I price these widgets at $475, to attract customers. It works; the widgets are flying off the shelves. And they're being purchased by people who used to be your customers, and I'm making even more money per-unit than you are.
What's your next move? Do you want to keep losing customers to me, or do you want to adjust your price to be more competitive?
Price wars are a race to the bottom that everyone loses. In reality, such oligopolies follow a kinked demand curve.
A new entrant isn't guaranteed to now price at $475. They'll see the incumbent being successful at $500. Now they price at $499 rather than trigger a destructive price war. Companies collude on this quite frequently. When everyone keeps their prices high, all get to enjoy the big margins.
Outside of that, ok so you have a warehouse full of widgets you need to move fast. So you undercut, and sell out. If demand is still bigger than your supply, you're now out of capacity, customers are going back to buying for $500 from your competitor. That means you've mispriced your limited inventory, so now you raise your prices up to closer to $500 because it helps you control your capacity, and also you know the market can clearly bear it.
Anyway, those are obviously overly simplified scenarios prices rarely fall down dramatically because of tacit collusion. Its asymmetric price transmission ("Prices go up like rockets, but fall like feathers")
Some lose more, i.e. PRC manufacturers well incentivized to involute to drive competitors out of business. $500 for 10% marketshare is less than 100$ for 60%. Of course PRC being spoiler, at least under current geopolitics where they have less reason to align with existing memory cartel.
I don’t have data for how much tvs are currently subsidized. But you can just look at the inflation adjusted price of TVs from say 1980 to 2010 and see the drop without worrying about adjusting for advertising and spyware subsidies.
You can also look at computer monitors (which don’t have advertising and spyware) and see an enormous price drop.
People keep claiming this but it isn't true. The subsidy from advertising is very low:
> They make about $20 per user annually and, assuming an active TV service life of five years, yield about $100 over the lifecycle of a main viewing room TV.
Privacy and Bot defense are opposite ends of the same fulcrum. If you permit privacy, the site/service has to trust users to behave and follow the rules. If you track users, then the users have to trust the site/service owners not to abuse that trust. There isn't really an in between.
So if you want privacy, you have to accept poor and sometimes insecure services.
Right now the only codebase I care about them fixing vulnerabilities in are the 3800 repositories that got stolen from GitHub.
"Vulnerabilities in the software that makes the internet" is honestly lower priority than "The platform that the software that makes the internet uses to make releases" If buyers of those internal repos find ways to break into GitHub such that they can cut software releases, or poison github actions from a distance, then we're all in a very ugly mess.
Don't forget that in those 3800 repos is likely also npmjs.org itself.
Would they do the same thing today or have an LLM re-implement those 3000 lines of c++ ?
reply