Hacker News

There was a presentation at CS 25 Transformers United about some phenomena like chain-of-thought emerging only after training LLMs with at least 1T tokens and only in LLMs of certain size.

https://youtu.be/tVtOevLrt5U?t=923

This is actually a well-known phenomenon in ML, called "The Bitter Lesson".

> One thing that should be learned from the bitter lesson is the great power of general purpose methods, of methods that continue to scale with increased computation even as the available computation becomes very great. The two methods that seem to scale arbitrarily in this way are search and learning.

The full essay is worth a read, it's pretty short http://www.incompleteideas.net/IncIdeas/BitterLesson.html

> one of the basic tenants of algorithm development was that you can't just brute-force your way towards a solution for some complex problems

It's kind of sad that popular CS textbooks often focus on solving precise problems with lowest theoretical complexity bounds while ignoring more practical (but generally applicable) computation techniques.

In machine learning they call it "gradient descent", which in older days had analogies in techniques called "hill climbing", "local search" and "simulated annealing". Basically you have a function you need to optimize for, and you clumsily tweak the parameters so that you get the (locally) max/min value you wanted. These techniques were great at finding approximate, locally maximal solutions without trying all the possibilities at once (which is more akin to the kind of "brute force" in the traditional CS context).

I guess because these techniques were generally applicable yet the outputs were approximate and you couldn't analyze them much (no fancy O(n log n)), the theorists did not find them interesting and thus were not put into the spotlight of student's learning curricula.

In modern machine learning they do this gradient descent thing which is also tweaking the parameters bit by bit to optimize for the loss function, except that the parameters are now in the billions and trillions. The compute required is huge of course, but it's actually quite an "efficient" process, and it's not actually doing much of "brute forcing" at all. During training, the process is essentially, almost equivalent to, compressing the many many trillions of tokens of training data. To me it's quite amazing that they manage to complete such a process within a couple months of training, even if they have hundreds of thousands of GPUs...

There is a Sanjeev Arora paper "A Theory for Emergence of Complex Skills in Language Models" (https://arxiv.org/pdf/2307.15936) on this subject. The key idea is there is cross entropy (how "surprised" the model is with the "correct" next token, lower is better), some of which is inherent in the language and therefore unavoidable, and the rest is model error, and that this portion of the cross entropy is reduced with scaling.

And as scaling reduces a model's excess entropy, the model can become good at combinations of skills much faster than you would expect if it had to separately see and memorize every combination. They call this "slingshot generalization".

It might be that what we consider a basic and very hard puzzle are extremely close together on a more absolute scale. The difference is often for us what proportion of humans can solve it. And the low end of that is still quite high up - animals that can solve things that are very basic for the vast majority of humans are pretty rare and known about, yet are capable of quite complex actions and learning and aren’t wildly different in scale of neurons to us.

Going from 1m to 1T params is also a scaling of a million times. It’s like going from a human brain down to one percent in size in each direction or just a few mm.

IANAMLE, but there is "grokking" that makes models learn to actually generalize, even after you give them enough parameters that would let them memorize the dataset:

https://en.wikipedia.org/wiki/Grokking_(machine_learning)

High-dimensional gradient descent behaves very differently than the simplified 3d visualisations we use to demonstrate it, and has lots of ways out of local minima:

https://www.youtube.com/watch?v=NrO20Jb-hy0

so it seems like there is a benefit to giving models more space to learn in rather than forcing them to compress the knowledge from the start.

> Like isn't it weird that the 1 million parameter model with the same architecture can't solve basic puzzles but suddenly the 1 trillion parameter can conjure up counter-examples for the Jacobian conjecture?

I'm not sure what you mean? You can see the intelligence of LLMs progress predictably and stably according to scaling laws. LLMs have to encode language in addition to intelligence so there's a minimum bound for them to output sensible text (you can train specialised tiny models to solve basic puzzles without language). Start at around 127M and compare models of increasing parameters and you'll see a clear progression in intelligence.

> It's unintuitive since, to the best of my knowledge, one of the basic tenants of algorithm development was that you can't just brute-force your way towards a solution for some complex problems, e.g. naive sorting algorithms suddenly won't beat quicksort if you put more processing to them

How is that a basic tenet? Simple, easier to parallelise algorithms that have lower memory requirements, or can take better advantage of hardware, or don't hit a plateau the more compute you throw at them, can absolutely beat cleverer algorithms. E.g. brute forcing rendering with Monte Carlo path tracing will give you more physically accurate results than ray tracing or rasterisation algorithms that rely on a bundle of hacks to approximate global illumination, transparency smooth shading, etc.

Here's one way it could happen:

Let's say there's some circuit that does problem solving of the kind we call intelligence.

We dont know what this circuit looks like, but it exists in our brain.

Doing regression on outputs from the brain (e.g. internet text) with enough parameters, we can "fit" our model to this circuit.

But if you try to fit it with fewer parameters than it needs, you're just going to get some linear approximation.

http://www.incompleteideas.net/IncIdeas/BitterLesson.html

> one of the basic tenants of algorithm development was that you can't just brute-force your way towards a solution for some complex problems

Mote-Carlo is pretty useful still. Not sure if your statement holds

it's certainly not a definite procedure for determining if an arbitrary mathematical statement is true or not. it's more like educated guess and check which definitely scales up

Is it just me or does this read like a re-implementation of LSTMs?

I believe this is the repo:

https://github.com/MoonshotAI/Kimi-Linear

... holy cow!

I'd kindly suggest that we could also stop calling them "distillation attacks".

False dichotomy right?

Are Chinese labs impressively innovating? Clearly.

However this doesn’t rule out possible gains due to distillation.

I don’t know the degree of the latter but both things could certainly be true.

The distillation complaints to me sound like when a casino complains about card counting

I stil don't understand them. I want the US to "win the AI race" but I have trouble understanding how most of all inventions today aren't "distillations" of past knowledge. Is Anthropic claiming the data they stole as trade secrets?

You can’t build a frontier model with one single thing. This is an incremental improvement but it doesn’t explain the entire success of the model. The training set is immensely important, regardless of how you feel about distillation.

It can easily be both. Also, they didn't use this innovation in K3 - K3 pre-training would have started months ago and the paper only mentions a 48B model. The people working at this level may not even be heavily involved in shipping a new iteration of K3, or at least theory contributions to it were done many months or even a year ago and after that it is all engineering.

Does one have to exclude the other?

Well said.

The distillation theory does not even make sense as Fable was only around for days (effectively) before Kimi was released.

For K3 read this instead: https://arxiv.org/abs/2607.24653

The main contribution of the K3 paper is Stable LatentMoE. Like some other models it compresses data sent between layers, which puts certain requirements on the router. K3 improves performance by using a more balanced expert selection strategy.

Rather under-discussed back then: https://news.ycombinator.com/item?id=45766937

I believe OP posted it because the new Kimi K3 has 69 KDA layers (the rest are 24 Gated MLA), I think previous large Kimi models had only MLA layers.