> tinfoil doesn't seem to distinguish between cached and input tokens.

Correct.

Privatemode correctly distinguish[1][2].

But with Tinfoil it appears to be a fundamental architectural limitation that they cannot.[3]

[1] https://www.privatemode.ai/pricing [2] https://www.privatemode.ai/blog/secure-prompt-caching [3] https://docs.tinfoil.sh/resources/caching

Hi, cofounder of Tinfoil here. We're working on implementing prompt caching. It just requires more work than with traditional, non-private inference and requires some careful considerations to actually implement it.