One of the most important graphs in economics shows a divergence between increases in labor productivity and worker take home share since at least the 1970s
What is being discussed is KV caching [0], which is used across every LLM model to reduce inference compute from O(n^2) to O(n). This is not specific to Claude nor Anthropic.
> How do you do "due diligence" on an API that frequently makes undocumented changes and only publishes acknowledgement of change after users complain?
1. Compute scaling with the length of the sequence is applicable to transformer models in general, i.e. every frontier LLM since ChatGPT's initial release.
2. As undocumented changes happen frequently, users should be even more incentivized to at least try to have a basic understanding of the product's cost structure.
> You're also talking about internal technical implementations of a chat bot. 99.99% of users won't even understand the words that are being used.
I think "internal technical implementation" is a stretch. Users don't need to know what a "transformer" is to understand the trade-off. It's not trivial but it's not something incomprehensible to laypersons.
They are caching internal LLM state, which is in the 10s of GB for each session. It's called a KV cache (because the internal state that is cached are the K and V matrices) and it is fundamental to how LLM inference works; it's not some Anthropic-specific design decision. See my other comment for more detail and a reference.
> 99.99% of users won't even understand the words that are being used.
That's a bad estimate. Claude Code is explicitly a developer shaped tool, we're not talking generically ChatGPT here, so my guess is probably closer to 75% of those users do understand what caching is, with maybe 30% being able to explain prompt caching actually is. Of course, those users that don't understand have access to Claude and can have it explain what caching is to them if they're interested.
Surely you jest? The software industry is in its current sorry state because of multiple generations of human developers happily producing an endless stream of shady features.
I have a theory, that the "FANG" companies pay such high salaries in compensation for making those devs implement shady features that are harmful to everyone except the bottom line of the company.
It's hardly a theory when the converse is plainly true.
Look up similar jobs for academia, government, or NFP/Charities. They're (on paper) driven by their mission, not by profit, and the salaries match that goal.
There is no shortage of game dev talent or adjacent creatives. It’s doubtful they will go on to find roles in the industry given the current climate and they presumably don’t have the capital to gamble on multiple years of game dev for potentially no return.
It's not a small thing to create a team who's proven to ship a successful offering, trust me I've spent a lot of time trying to figure out how to do it and hitting that bar is hard
Honestly as someone building a game right now, I'd love to meet some people who are trying things and are open to meeting
I guess the argument would go that a new economic model will be required at that stage.
There isn't much point in having people do jobs they don't like which are trivial to automate just for money, but at the point where there isn't enough economically useful things for everyone to do, the current system falls down.
> What is the benefit exactly?
Well one benefit would be international competitiveness. The country that does it slowest will be the country doing more work for less output.
> but at the point where there isn't enough economically useful things for everyone to do
This assumes that for example a person who has been an artist for 20 years, can easily enough switch professions to a machinist, and the only reason for them not to do it is because the economy has no need for another machinist. An insane way to think. This is not how humans work.
Let me see any HN dweller go from their cushy home office to butchering animals for meat on 12-hour shifts for example... Oh and btw, no safety net to give you food, housing and healthcare while you learn the new craft!
> Let me see any HN dweller go from their cushy home office to butchering animals for meat on 12-hour shifts for example
I think that's the reality of lots of people when they face any redundancy situation - People take up jobs that they wouldn't traditionally want to do in order to survive or look after their family. I don't necessarily see why people on HN would be different.
> I guess the argument would go that a new economic model will be required at that stage.
> ...but at the point where there isn't enough economically useful things for everyone to do, the current system falls down.
Not necessarily. To quote the Bobs from Office Space: "He won't be receiving a paycheck anymore, so it will just work itself out naturally." No need to change, just let the plebs die out.
As of now, there is no benefit to regular working people.
Perhaps in the future, great abundance will occur, but as of
now, there will only be job loss, fear, neo-luddism, and blame.
Believe me when I say that I know people, some close to me,
that are experiencing fear due to automated systems being
installed and tested where they work. They are essentially
witnessing start of their automated replacement robot
workforce.
Whatever is planned in terms of AI being used to help people
needs to happen, sooner rather than later, because all I
am seeing is chaos in the horizon.
I think this is a relatively unique outlook and not one that is shared by most.
If you use a tool to automate sending emails, unrelated to LLMs, in most scenarios the behaviour on the receiver is different.
- If I get a mass email from a company and it's signed off from the CEO, I don't think the CEO personally emailed me. They may glanced over it and approved it, maybe not even that but they didn't "send an email". At best, one might think that "the company" sent an email.
- I randomly send my wife cute stickers on Telegram as a sort of show that I'm thinking of her. If I setup a script to do that at random intervals and she finds out, from her point of view I "didn't send them" and she would be justifiably upset.
I know this might be a difficult concept for many people that browse this forum, but the end product/result is not always the point. There are many parts of our lives and society in general that the act of personally doing something is the entire point.
It doesn't sound like utopia to me, hence the quotation marks. Eminently achievable, but not actually good. Only those engaged in utopian thinking - with a heavy slice of ignorance of basic economics and history - would think it is utopia or leads to it.
