From 300KB to 69KB per Token: How LLM Architectures Solve the KV Cache Problem (news.future-shock.ai)
31 points by future-shock-ai 3 days ago
az09mugen an hour ago
Unrelated, but 69KB is how much RAM Voyager 1 has.
31 points by future-shock-ai 3 days ago
Unrelated, but 69KB is how much RAM Voyager 1 has.