Smaller, faster, safer: running Kimi and GLM at scale (blog.cloudflare.com)
61 points by ascorbic 3 hours ago
scrlk an hour ago
Nice to see a provider being transparent about KV cache quantisation. I've been suspecting that some providers do this silently whilst heavily promoting their unquantised weights, even though KV quantisation can degrade quality more than weight quantisation.
However, I wish their testing were more detailed. Firstly, some model families are more sensitive to KV quantisation than others (only Kimi K2.6 was tested). Secondly, the evaluation suite they use to claim that FP8 KV quantisation is indistinguishable is noticeably lacking coding benchmarks; in long-running tasks, minor tool call errors compound over time.
syntaxing 34 minutes ago
> View pricing in the Cloudflare dashboard ↗
Why… I wanted to see if it’s worth it to use cloudflare’s endpoint but I can’t even see the pricing
brokenodo 2 hours ago
I was interested in reading this until my slop detector went off at the paragraph starting with “It's worth being precise about where the benefit comes from, because it isn't raw speed.”
I love AI, but I really hate reading it.
hankbond an hour ago
I have had to stop commenting this because it would end up on 50% of the posts here. I really wish we could flag prose as ai-generated on here and just filter it out.
dgellow 12 minutes ago
Don’t stop commenting about it, if there is something we (the readers) can do is ensure it is seen as uncool to post slop content
gr_norm an hour ago
LinkedIn (of all places!) announced a button for flagging this recently: https://www.linkedin.com/posts/hsrinivasan1_ai-slop-is-a-top...
How well it would work on this site, I'm not sure.
speedgoose 44 minutes ago
hankbond an hour ago
physix an hour ago
trollbridge an hour ago
Sign up for Pangram and install the browser extension; covers X, Reddit, and Substack, and more to come.
arjie an hour ago
Cloudflare blogs are not meant to be human-read, AFAIK. They're raw material meant to be fed into an agent to be filtered down. I rarely read the contents because they are usually word-expanded to a greater degree than an article from The Atlantic.
dgellow 11 minutes ago
That’s disappointing, in the past cloudflare had some of the best engineering blog articles