A line-by-line audit of how KV-cache growth drives decode-phase cost per token, with batch effects and context-length penalties. For teams serving LLMs in production.
Visit pastagi.comPublic thread — visible to everyone. For private notes to the poster, use Private feedback above.
Sign in to join the discussion.