At Revefi, we made an interesting mistake with prompt caching for our agent - adding the current timestamp into the system prompt. We solved it by adding it to the suffix. At first, we didn't think much about it - but then we started seeing the same mistake with our customers' agents (we work in the AI tokenomics space). So we thought we'd share our 2 cents about it.
I know there's a lot literature out there about the theory of prompt caching, but hopefully an explainer on how we did things wrong and how we fixed it will be insightful to read.
At Revefi, we made an interesting mistake with prompt caching for our agent - adding the current timestamp into the system prompt. We solved it by adding it to the suffix. At first, we didn't think much about it - but then we started seeing the same mistake with our customers' agents (we work in the AI tokenomics space). So we thought we'd share our 2 cents about it.
I know there's a lot literature out there about the theory of prompt caching, but hopefully an explainer on how we did things wrong and how we fixed it will be insightful to read.
https://www.revefi.com/blog/how-revefi-reduced-its-data-agen...