For Deepseek V4, The main effect of raising the reasoning effort is to add a little section[1] to the end of the system prompt that says "BTW make sure to think really hard! :)"
If Anthropic's models work the same way, then changing reasoning effort would break the cache because the API has to modify the system prompt given at the very start of the context and rerun the whole thing through the inference server.
This kind of limitation is one reason Opus 4.8's mid-conversation system messages[2] are actually a pretty big deal (if they actually work).
> This kind of limitation is one reason Opus 4.8's mid-conversation system messages[2] are actually a pretty big deal (if they actually work).
Didn't they start injecting system messages telling Claude to calm his tits in overly long and emotional (iirc it triggered on some keywords) chat contexts last year?
The model is trained to treat system messages differently from regular user/assistant messages. Most models are trained to only expect system messages at the start of the conversation. This is changing now.
If Anthropic's models work the same way, then changing reasoning effort would break the cache because the API has to modify the system prompt given at the very start of the context and rerun the whole thing through the inference server.
This kind of limitation is one reason Opus 4.8's mid-conversation system messages[2] are actually a pretty big deal (if they actually work).
[1] https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash/blob/ma...
[2] https://platform.claude.com/docs/en/build-with-claude/mid-co...