Hacker News

People have told me I was smart since I was a kid, but I can't remember for shit. I had a thought when I was fairly young that the only reason I was (maybe, sometimes) outperforming others intellectually is that I was habitually compensating for my poor memory by working things out on the fly, while others could rely more on rote memorization. Anyway, takes all kinds I guess!

In line with the other comment on this post: I also am classified as “brilliant” by people and IQ tests. My country gave me an award for being a great inventor. But my memory is worse than anyone I know.

I can’t remember the lyrics to any songs, or the lines from any movies, or what I’m doing this week. I don’t think it’s a strong hypothesis.

My wife has a nearly eidetic memory, so marrying her was a game changer for counteracting mine :)

> out-remembering people around us.

Doing an angular -> react refactor with Opus 5 at work.

Amazed at how good it is but also amazed at how dumb it is after I told it I want a very basic and specific code review:do the file names match the contents? is one functionality/concern split in multiple files? do the variable names match what they do? does a method read nicely?

Anyway, I was very surprised at how bad it can be at this.

I suspect it's because the LLM simply does not focus well enough on a single method.

My gut modelling is that the critical files are quite small, some 4k LOC and the LLM tries to read them all in one go and my requirement is surprisingly difficult but the LLM does not realize it, loads all the input into context and simply cannot find a good signal per token/group of tokens.

I guess I can do it because I literally cannot fit 4k LOC in my head and I actually read only tens of lines but I can reeealy dig deep in those lines.

The LLM can remember lots, it has 1M context window, was trained with needle in the haystack problems, but it can severely underestimate how much computation is required with TASTE/universal computation type tasks.

Yes, LLMs are incredible when it comes to delivering well defined outcomes, it's evident they were RL trained very well. But they are incredibly bad at understanding nuance in a text.

It's incredible how agentic coding benchmaxing seems to correlate with writing good poetry but at the same time finding good names for files and variables and wiring beautiful code does not.

Same with getting basic details of who said what in an "adversarial" conversation between 2 humans -- LLMs are incredibly bad at it, with all their "almost discovering new maths" skills

Have done a lot of “parallel invention” where I invent something genuinely novel & new-to-me, only to later discover it was first developed eighty years ago.

The part that’s really disheartening is there’s so much knowledge out there, I wonder if this is actually efficient - maybe you can’t reasonably search the state of the art across all of human knowledge and so you have to invent it again. But that’s such a sad outcome.

Memory is one aspect of intelligence, but hardly the only one. Making new connections, logical deduction, spatial intelligence and creativity are also aspects.

I have fairly poor memory, but I can solve problems that people with better memory can't.

> I don't think I've had a truly original idea in my life. Combine A + B, when it's rare for people to know A and B at the same time.

There is also something to be said about recognizing that A + B can be combined.

And also in accurately judging that other combinations will be useless and avoiding them.

Since it is so much easier to test for memory than understanding society grew to confuse the two. Only now faced with these other entities are people beginning to question it.

I know people that got to post grad math without understanding a thing but they could remember a lot easily, while many of those that understood but had a harder time remembering every last variation of everything got penalized.

Human can have periods of heightened mental capacity. Sometimes I looked back at some of my work days or weeks later and amazed how I could have done it. When you're in the flow, have enough sleep, and have long period of non-interrupted time, you can do amazing work.

There is an ability to hold ideas in the mind simultaneously. Some can hold a great deal more than others. It can be exercised, but is definitely bound by genetics.

If you want an example of someone at the near peak of human ability, check out Jon Von Neumann.

Then there is an ability to peer deeply into complex problems and somehow find the simplest truths that make sense of it all. Think of Einstein.

Both are incredibly intelligent, but in different ways. I'd say Von Neumann's memory was far greater than Einstein's though.

One can flawlessly ponder anything known to man, and the other could ponder completely original ideas (to an extent)

I like to use the analogy that the human brain (when it comes to intelligence) is like a computer. We have storage, which is just long-term memory. We have RAM, which is your ability to keep track of a mental model of something you’re actively working on, and then there’s the CPU, which is the ability to make logical leaps and connections on that mental model (or maybe storage).

I’ve met different people throughout my career whose intelligence came in 1 specific area. For instance, my friend is extremely good at trivia, he clearly has a lot of storage and can access it easily. I think I’ve only met one person who was excellent in all three areas of intelligence.

Obviously this is a simplification, but it’s how I like to illustrate my ideas on intelligence at parties and first dates.

> I don't think I've had a truly original idea in my life.

There’s no such thing as a truly original idea. It is all just combining A+B!

> just bringing back random knowledge from previous jobs or self study, and being able to apply it to the problem at hand.

Being smart, at least in the context of the workplace, is about being checked in to whatever you’re doing, and drawing connections across your experiences.

There is also an element of how you process information. Some people jump to instantly refute an idea with a counter example. Some people jump to statistical likelihood. There’s elements of trust vs trust but verify.

Some people, through practice and experience are better than others at breaking down foreign information and making sense of it, or jumping to an assumption.

I also think there’s an element of just remembering what my final conclusion was on an opinion, vs all my old opinions and all the work to get there. It’s easier to just store current belief and re-work-it out how I got there later.

People who study intelligence differentiate between fluid intelligence (plastic intelligence) and crystallised intelligence. The former is the ability to learn and form new ideas. The latter is - as you said - memory. Knowing lots of stuff.

They’re positively correlated. Presumably because people who have good plastic intelligence often end up learning more throughout their lifetime. The only way to form knowledge in our minds is by learning it.

Possible, but not true. Much of intelligence is also creativity. Otherwise we would call Google search intelligent. Strong recall is a different kind of intelligence, and good memory is indeed a positive trait. But it is not what we generally call intelligence on its own.

This sounds very much like the joke about economists finding a $100 bill. In reality there is innovation and new ideas. Humans discover new things and change their behavior. So your theory needs to also accommodate that fact.

Agreed “remembering” stuff is critical in most life’s situations where if you did, you will appeared to be smart. This is probably why spaced repetition has been a key element in learning for the past few decades.

But “remembering” is only the beginning (you have to remember first!), after that, elements like understanding relations, connecting dots and remixing, timing, etc will truly make one shine.

In one way, it’s like the current “LLM + Harness” setup for agents. LLM is how well it remembers, but different harness techniques really matters, at times even a worse model mixed with great harness can outperform great model with bad harness

Another ai article, another group of self-identified smart people claiming they are not as smart as a statistical engine. Are most ideas not original sure. Was everything idea discovered first at one time or another sure. Maybe what's going on is you are not as smart as you think you are and intelligence is less proprietary than you believe. Nobody owns ideas. What matters is if you know which ones to use. There is a distinction between knowledge and intelligence and between epistemic effectiveness and interpolation about the mean of the corpus. If ai wows you likely that's because you are actually more average than you believe. I have used AI a lot. Never once do I think this tool out thought me. Because it doesn't think at least in a human way at all. Being able to retrieve some information or connect some dots is less meaningful than knowing what matters. Ai needs vast working memory because it lacks endogenous constraint driven judgement. Whereas humans historically lacking judgment died and now more often they just make fools of themselves online.

I suspect any new thing created/discovered/etc is just combining knowledge from existing things with memories being an example. Outremembering while certainly not the only method helps massively with combining knowledge from existing things. I don't see new knowledge coming out of nothing.

The amount of times I’ve invented something genius in software dev to later discover it has a name…

> I suspect that a lot about what we call being very intelligent is ultimately out-remembering people around us.

I wonder practically how useful it is. It should be very useful, right? Say you're a hiring manager, and now you can hire thousands of students who have almost eidetic memory across pretty much all the subjects. The students can produce amazing work if you give them specific instructions. They are also very good at mechanical pattern matching. That is, they are the uttermost crammers. They are like the students who don't really get STEM yet they could, through memorizing all the problem-solving patterns, ace any national college entrance examinations, IMOs, PHOs, Putnams, you name it, and all the way to solving hard problems like Sendov Conjecture - they even found out elegant elementary solutions! Mind you, Terrance Tao proved a weaker conclusion when studying the Sendov Conjecture and got to publish his work on The Big Four. That said, the students won't take initiatives, and they sometimes make very silly or even impossible mistakes, to the point that it requires your supervision and review. Would you hire them to do your company's work? I'd assume that answer is a resounding yes.

Could just be that you aren’t an ideas guy/a lateral thinker

I feel like you can make the same analogy with products as well.

It’s a false definition of intelligence - remembering stuff has never been what intelligence is.

Understanding especially in the context of unknowns is what intelligence is.

Not only I agree with you that I don't think I've had an original idea, even when I try to do artistic stuff it seems like the things most praised were the ones where I was trying to do something else and the good part came from failing to do the thing I was trying to do.

why would anybody believe true brilliance? They would only believe something that can be proved from combining existing knowledge.

If a time traveler went back to 1600 and started spouting off about differential equations everyone would think them quite mad.

I think that’s true for genius is in general. I don’t mean the Einstein type but I mean the child prodigies that graduate high school at 10 years old or whatever I mean if you can read something once and remember virtually everything about it that’s a gigantic leg up on everyone else who has to study drill the stuff into your head, etc.

Even in a debate, if somebody just has the ability to remember tons of facts and figures, the other person will seem unintelligent by comparison, even if the other person is correct

I've long held the belief that humans cannot truly create. We just remix and recombine. This is interesting - although I'm not religious, many religious texts call god the "Creator", perhaps because the ability to truly create original thought is above the ability of us humans. To truly create is divine. Maybe the writers of those old texts recognized this as well.

> Combine A + B

Really, compress. Compression is a useful proxy for intelligence.

To compress data efficiently, an entity must discover the underlying patterns, structures, and regularities within that data. Identifying these structures is precisely what we define as "understanding" or "intelligence.

Anything we put into writing can be thought of as a string of data. The Kolmogorov complexity of a string of data is the length of the shortest possible computer program that can output that string.

A good compressor doesn't just memorize (yes, there's a reason LLM writing features the "it's not just X, it's Y pattern"), it builds a generalized model to handle unseen, similar data. LLMs are trained by compressing the internet into weights, they use this to write new code.

When an LLM is trained, it compresses ...bytes of human text into a few ...bytes of neural network weights. When you ask it a question, it uses that compressed model to generate (decompress) a coherent response.

Consider the dichotomy:

LLMs display emergent reasoning, coding capabilities, and logic because they were forced to compress human language so tightly.

LLMs often suffer from "hallucinations" and struggle with novel logic puzzles because compression encourages mimicking statistical patterns rather than building a true causal model of the physical world.

An algorithm can achieve massive compression if it utilizes a highly complex, slow, or massive decompression engine. If a system takes 10 000 years to decompress a file, or requires an infinite amount of computing power (like Kolmogorov complexity), it cannot function as a practical intelligence. Real intelligence must operate within tight time and energy constraints.

> Maybe there's true brilliance out there

Brilliance is a trope. pg's 2009 "The Anatomy of Determination" deals with it: https://www.paulgraham.com/determination.html

In it, he argues that determination is the single most important predictor of success, surpassing even raw intelligence.

While popular culture loves the narrative of the "brilliant genius," Graham's experience backing hundreds of startups taught him that a highly intelligent person without drive quickly becomes ineffectual. Conversely, a profoundly determined individual can find pathways to success even in ordinary or unglamorous industries. Over time, determination actually starts to look like talent.

I'm going to commit a faux pas (a breach of HN etiquette) and give an AI summary in 3 bullet points because, well, the article deals with 3 points, so it's a good fit and I doubt I can compress it better than AI:

1. Willfulness. The desire to get your way regardless of obstacles or consequences.Being fearless about consequences and pushing through "hellacious" moments when others quit.

2. Discipline. The ability to force yourself to do what you planned to do. Being "hard on yourself" and turning off the background analysis in your mind that questions the plan.

3. Ambition. The grandness of your destination and what you want to achieve. Choosing the right target; if determination is a vector, ambition is the direction it points.

Of these 3, it's quite obvious at this point that LLMs are superior at 1) and 2), as LLM alignment is quite good, and it's ability to spin up sub-agents that can work for over 24 hrs is better than humans. Human limit is basically 1 hackathon's worth of cognitive work. Beyond that, health becomes a concern, and how many people can produce useful work under sleep deprivation? Codex can work for 24-ish hours, as evidenced by today's "Auto-research with codex: How I achieved a 232x Faster Kernel": https://news.ycombinator.com/item?id=49309549

So what's left for mere mortals? Apparently, nourishing your ambition:

"So here in sum is how determination seems to work: it consists of willfulness balanced with discipline, aimed by ambition. And fortunately at least two of these three qualities can be cultivated. You may be able to increase your strength of will somewhat; you can definitely learn self-discipline; and almost everyone is practically malnourished when it comes to ambition."

AI has sitzfleisch

https://en.wikipedia.org/wiki/J._Robert_Oppenheimer#:~:text=...

Ever heard of string theory.

People go whole lives without being able to make it pan out.

Also out-speeding them, and that was before high speed inference.

Out-ralphing them, you might say!

https://ghuntley.com/ralph/

AGI ≈ artificial stupidity × infinite persistence

> It's also "out-brute forcing them."

That is also approximately what people have always done to succeed.

Mathematicians routinely spend years on a problem without getting anywhere.

The key here is that it’s depending on the human inability to connect the sum of relevant knowledge, but said knowledge comes from humans.

Theres going to be this field day of low-hanging fruit that ML can round up, but after that I suspect it will be in fits and starts as a “connection maker” rather than some proof producer.

Counterpoint: something has happened in frontier models, and yes they now get discouraged and will sometimes prefer to not continue working on a problem unless you tell them to anyway.

I don't know how or why this would be trained on behavior, but no, it isn't true anymore that models don't say things like, "Ugh," or "this is going to take hours and maybe we should stop here."

> If a mathematician picks a research direction and spends a whole week on it and it doesn't pan out, they will likely be annoyed, need a break for a while, etc.

Your timelines are a bit unambitious. There's nobody expecting to make significant progress with a week of work.

We're just going to slowly deconstruct every element that could be a factor of intelligence.

It's not out-thinking, it's just out-remembering

It's not out-thinking, it's just out-working

It's not out-thinking, it's just able to consider more things simultaneously

It's not creative, it's just randomly generating things and then selecting viable ones

In other words: Thousand monkeys with a thousand typewriters...

https://news.ycombinator.com/item?id=48231974

The bandwidth is absolutely there ("we tried this and it didn't work" is totally the stuff of conference discussions).

The incentives are not.

The incentives are skewed towards "a magician never reveals her secrets". The results are presented as if a rabbit got pulled out of a hat, with a maximum ta-da! effect, and little backstory of how the hell did we get there.

Don't get me wrong, these things are discussed, often over beers (you better drink it you want to make a career in the field).

But not published.

The younger mathematicians are trying to change that with the blogging culture. But the professional incentives aren't there. (In corp-speak: can't put blogging on perf). They burn out.

That's why math blogs usually come from either the top dogs in the field, like Terrence Tao, who don't need to care about perf, or people outside academia.

That's one thing that I hope the disruptive/destructive effects of LLMs will force mathematicians to face.

As one of my fellow mathematicians sarcastically wrote¹, we've reached a point where we should become a cult because we're acting like one anyway.

The other possibility is, of course, that the shake-up will take us precisely into that direction.

My point here is that the real problem here is not mathematical; it's a social one: incentives and politics, organizational structures, policies, allocation of jobs and funding.

All of this directly impacts how we do mathematics, who we do it with and teach it to, how we teach and communicate, and, of course, what math we even do and look at.

Given that, I'm neither too worried about humans vs. AI standoff, nor hyped about the Glorious New Future full of AI-assisted discoveries.

AI or not, the organizational issues in the field are still there, as are the incentive structures (including the infamous publish-or-perish).

We are doomed, yes, but by our own hands and committees. And it's up to us, not the AI, to get us out of there.

The little shove from the AI might be just the thing we need.

____

¹ https://www.mcsweeneys.net/articles/an-open-letter-to-the-ma...

Yeah, LLMs are great at generating negative results for math-related prompts "We scanned values {a,b,c} from 0-100 and no results" Great..too bad journals will not publish this. But good job, I guess. A negative result is only truly useful if it can be bounded, requiring an actual proof.

What is next years tsunami of change? Asics?

> many top mathematicians have, through hard work, internalized many more complex mathematical chunks than ordinary humans

Do you really think an average person can internalize complex math? Them compressing it effectively and then remembering it is a sign of (very) high intelligence.

LLMs are enormously good 1. and 2. But humans are much better at the rest.

I don't want to read another LLM documentation ever again. The information to text ratio is way too small.

> That's all thinking is

That's more because a PhD (especially in this socioeconomic era where academia is also a microcosm of capitalism) does not automatically teach or focus on deep inquiry, rather narrow research programmes. Case in point, fallaciously reducing it all to a superficial theory of tricks means that by the same light, general relativity or any other major scientific result is merely just a trick. That framing ought to be absurd on the face of it but the fact is many PhD students do still graduate with very unexamined understanding of science, cognition, philosophy, etc. (Another way to see this is to note how such an argument is letting the notion of "trick"/"heuristic" do all the heavy lifting--something an advanced education should have nurtured the critical thinking skills to avoid that very metacognitive pitfall in the first place, which is ironic.) (Yet another way is to see how increasingly experts of one field or another insist that one must stay on their lane when speaking about a complex topic.) There are professors and thinkers who have written/spoken about this but they are a minority. Even Einstein himself when he complained that more and more scientists were trained/structured to miss the forest for the trees, and that was almost 100 years ago.

For greenfield projects LLMs don't need abstractions, but as the project gets more complex, the right abstractions save a pot on input tokens (less code to read) and reasoning tokens (less work to do to figure out the code), so they free the context window for higher purposes

Also I suspect that, apart from that, the results on smaller, cleaner codebases are better. LLMs degrade when following more than N instructions (where N depends on the model) even if the context window is not full yet; I suspect they also degrade when code has too many unnecessary concepts and details

I've been working on generating a large code base for the last couple of weeks. Finally got around to generating a sort of code-duplication report and have spent the last week just having it de-duplicating logic that had been strewn all over the place (eg 11 different functions all doing date math to add x days to a date). dozens of items that had each been similar functions duplicated numerous times. crazy. (opus-5-utracode)

This is the exact opposite of what I’ve been dealing with for awhile. LLMs absolute cannot work on something without an understanding unless they can outsource the understanding to a verifier. If you’ve got an easy to check function to measure progress then “keep going” is all the prompt you need. But if you need it to figure out “I pushed the up button and it moved up and left” then it’ll find the same bug five ways without realizing it’s just one bug in the underlying math.

LLMs use abstractions a ton in code though: standard library functions, popular libraries, etc. They just dont always make their own abstractions. At least not particularly good ones. LLMs work really well when they have well abstracted pieces to put together.

I also find them bad at what I call "abstraction compression." They're really bad at noticing when a helper function is needed, when some structure they repeated five times slightly differently can become a struct, when a whole section of code can be encapsulated in a simpler design.

I'm lucky that for my side project (an interpreter) I've written all the code myself, so I've built up its design in my mind over the past year, and so as I mull over what I'm writing I start coming up with simpler designs.

Interestingly using Opus 5 (and LLMs in general) has made me worse at this, since I don't feel the pain of writing something over and over again. On the other hand, I don't really want to implement a whole stdlib, so I have it write more of the auxiliary code. The hardest thing is that I have to manually manage the context, which is painful when I personally remember every helper function and why. I have to remember to keep the list of helper functions updated, which is irritating.

Interesting fellow. He has publicly claimed that he has ESP:

https://www.splcenter.org/resources/hatewatch/wikipedia-wars...

https://openpsych.net/forums/18/thread/25/?page=1#124

Gee, 100%. Articles basically saying genetics explain everything.

Depends how you define intelligence. I struggle to consider LLMs actually intelligent, at least for any definition of intelligence that I would be content by.

Is this assuming or studying human maintenance?

Takes the rogue having access to the $$$$ models though.

People aren't ready to prove/disprove every one who access to LLM and claims has changed course of humanity

We don’t trillions of dollars in LLM investment to build things mathematicians don’t understand. We already have plenty of those, even from ancient times.

As to your second point, Terry Tao already has an answer [1]: the proof isn’t the contribution, shared understanding is. This issue was already raised back when the four-colour theorem was proved. Machine proving and machine proof checking are useful tools but they don’t mean anything without the interpretative work and the communication necessary to build shared understanding.

[1] https://news.ycombinator.com/item?id=49056620

The entire point of writing proofs is for advancing human understanding. A giant dump of symbols that passes the lean compiler is meaningless besides human beings understanding it.

> The age of humans comprehending things is coming to an end: our brains just won't have the capacity to make meaningful contributions to science, math, or technology.

That sentiment makes me cringe. If you understand how LLMs work, you'd know it'll never be possible without a fundamental change in how these work.

We're also supposed to be reaching that point, somehow, without the LLMs ever being intelligent (in the dictionary definition sense, not the "high reasoning model" marketing sense).

Based on observations, the ones who are fooled by the supposed emergent properties, are just that, fools. Any sufficiently unintelligent agent will perceive transformer based LLM text predictors as possessing high intelligence.

"The age of humans comprehending things is coming to an end"

That's something AI companies would really want you to believe.

You’re prescribing elegance to a stochastic generator trained on the wealth of humanity, including 4chan. Let’s set our expectations a bit.

> It's going to produce proofs far more intricate than humans can understand, outdoing humans and opening new frontiers.

I agree.

> The age of humans comprehending things is coming to an end: our brains just won't have the capacity to make meaningful contributions to science, math, or technology.

I don't know if I see this being true for quite a while, if ever.

In limited experimentation: AI will certainly make statements that are extremely intricate and hard to understand, in part because they're overcomplicated and in part because they use a bunch of unnecessary terminology.

This is not to say that a human couldn't understand a streamlined version or that the AI would not be better if it made more streamlined statements to begin with.

(I am not saying that everything mathematical that an AI produces is in any sense trivial.)

It’s possible, but there’s a difference between vastness and difficulty.

Humans can’t compete with AIs on vastness of material they are familiar with, or the depth of effort they are willing and able to throw at a problem.

But scale isn’t the only aspect of difficult scientific endeavours. There’s also theory. And advancements sometimes come through hard graft of knotting together many things. And sometimes they come through the revelation of a deeper truth, or a new framework, a fundamental insight.

AI might help us reach the next level. But that doesn’t mean we won’t understand anything. It could be we have periods of vast intricacy we cannot follow, punctuated by profound elegance we (or at least experts) relatively easily can. And then the scaffolding we needed to get there falls away.

You could be right, but you're making a lot of assumptions about how complexity, scientific understanding, and explanations scale. One of the features of a good scientific discovery is that it often simplifies and compresses things that were previously a bunch of scattered facts. Also, as AI systems improve they'll get better not only at making scientific discoveries, but also at producing understandable explanations.

Then it's pretty bad that LLMs don't understand anything.

They don't know and can't know. Without an external source of input that corrects them, their output can never be verified.

If and only if that is actually true, then perhaps nothing matters. Until then, calling out shenanigans remains a noble art.

> The age of humans comprehending things is coming to an end: our brains just won't have the capacity to make meaningful contributions to science, math, or technology.

Ask yourself: is that really the world you want to live in? It's a world where people, all people, are sidelined.

I think the happy ending of that path is something like Idiocracy. And the more likely ending is something like "automated capitalist economy without the people, because the people couldn't compete."

That maybe true at some point, but i don't think we are there yet.

But apparently we can teach machines to do it for us

>produce proofs far more intricate than humans can understand

Math is not magic, a proof is just a series of applications of a set of rules on some axioms. A mathematician could understand any proof given enough time to study it; the only way for AI to make proofs that a human couldn't understand is by making really, really long proofs.

Why would you want something you don't comprehend? How can you be sure it empowers you?

I think perfect rationality doesn't exist, because it is rational to reject something that you don't understand. So rationality of a given physical system will always be bounded.

It's going to produce proofs far more intricate than humans can understand

The thing is, some number of advanced proofs start out "too intricate for most mathematicians to understand" but many of these get rephrase and reframed until they're accessible to undergraduates. Hopefully, AI math can be guided to do that sort of reframing to increase the level of accessible math as well as extend the border of math.

Let’s not jump the gun. Where they are right now they can somewhat match our abilities. We haven’t even gotten to the point where they can self improve.

Where would you get enough pre internet tokens to train a near-frontier model?

Mathematics is typically concerned with "proofs" [1], which similarly to code, often allow for strict validation. Thanks to reinforcement learning techniques, it is now possible to train LLMs to perform very well on code generation, and mathematical proof generation.

Law and medicine are fundamentally harder fields to obtain decent training data for, and LLM results are therefore expected to be less powerful. Also, making mistakes in these fields is costly, but perhaps you were alluding to that already.

[1] https://en.wikipedia.org/wiki/Mathematical_proof

A neighbor of mine whose husband is a lawyer said it's already part of his regular workflows. OpenAI also already have HIPAA compliant offerings targeting healthcare uses, etc. Of course they already do these things.

They try to sell AI as lawyer or doctor replacements as well. But because it’s HackerNews we are biased towards our domains to see them more often.

They can rely on compilers, solvers, theorem provers to validate the generated softwares and maths. That’s what makes it possible to iterate quickly in a loop and self correct. You cannot do that in soft industries like legal and medicine

Look up OpenEvidence

This is why we have hierarchies of abstraction. Pretty much every field of mathematics relies on constructing notations, models, and other tools to simplify things in a way that is verifiable. LLMs rely on the same basic technique, they can just pull from a wide variety of these abstractions at once. So far we've been able to understand their proofs just fine. Computer-assisted proofs in the past that relied on brute-force is where we have run into trouble. We cannot reason about millions of possibilities at once, and we had to trust that the computer program that analyzed them was correct, which is a really hard problem and leaves humans fairly unsatisfied. I think we are actually progressing in terms of understandability in computerized proofs.

We offload working memory to paper if we want to understand something that does not fit into the regular meat bits.

>Cause once AI makes arguments that require a working memory of hundred items, then we as humans will have no way of understanding the arguments…

That doesn't follow. We could still understand it just by studying it and committing it all to long-term memory, it just takes longer. And there's a hard cap on the working memory of LLMs, due to the quadratic scaling cost of the full attention layers that have proved unescapable for all SOTA LLMs.

“is obvious” -- that’s what my Russian math professor said in college before skipping the rest of a proof.

But was it?

Euler would like a word. Maybe it's our institutions.

Well, some folks are going to keep trying to exclude it, but the reality is we are almost at the point even those folks would concede that they are not smarter than LLM. I suspect we will see then see more discourse about consciousness, morality, agency, emotions as being the most important part of intelligence.

For instance, Opus 5 yesterday critiqued my resume and mentioned a ridiculous little detail-- my phone number area code didn't match the state in which I currently work (I know, I should anonymize but I couldn't be bothered). It failed to notice that I both worked in (previously) and studied in the state of my area code. This was just two pages of text, set to highest effort.

Why would you want to replace your lawyer with a set of tensors that does not actually think and makes mistakes like this? Lawyers tend to get hired in high stakes situations. Why wouldn't you instead say that this would be a great tool for lawyers to use judiciously in researching precedents, etc?

I don't understand what people are doing with models that makes them assign agency or intelligence to them. When I manage to forget the financial fuckery of the AI buildout and its implications, when I manage to forget scaremongering by loathsome CEOs, I still have the same fascination and excitement at the idea of LLMs as I did when I was playing with the GPT API prior to the release of ChatGPT.

LLMs are, to me, truly amazing tech. It's so fascinating to me that they now DO have emergent properties that look at face value like reasoning and intelligence. But every day that I work with them, I am repeatedly clobbered over the head with the fact that they do NOT reason and are NOT intelligent.

Why can't we be fascinated by emergent properties of intelligence without immediately jumping 10 steps into the future and, like a limit in calculus, assume that "this is it-- we're on the cusp of AGI"? To me, the fact that LLMs can combine existing ideas that people hadn't thought of combining in solving a novel problem is extremely cool. But my first thought is-- this is an amazing new tool for mathematicians and researchers. Instead, most everyone seems to jump the gun to the "humans are obsolete next year" conclusion.

I don’t understand why this is such big news. OpenAI has essentially made a bunch of marketing copy by gussying up an algorithm being given a near unlimited budget to stochastically permute through its lossy memory.

I think the real scandal is that we are almost 3-4 years into this (I think the release of GPT 3.5 is a good marker of when this public frenzy started) and all we’ve seen is OpenAI and the other major AI frontier companies constantly retracting their preposterous claims every time. We appear to have reach a local maxima in that it has some value in places that tend to be a little easier to scope and limit (computer programming, mathematical proofs). So, given the actual useful economic value this has provided, does this justify the investments? I think we are approaching 1 trillion in CapEx for AI [0]. For context, I believe the annual GDP of Norway is $600 billion.

[0] https://www.fool.com/research/ai-companies-spending-on-data-...

That’s how HN works, I had that multiple times over the years with my own submissions. Sometimes it gets picked up quickly, sometimes not. A post can also down rank very, very fast. It depends a lot on the level of engagement and the type of engagement

Follow the money.

How fast the upvote happened?

100%. Context is big for AI, but it's nothing compared to everything a human can learn. If you efficiently represent everything in context, it may be many papers, but if AI is actively working through proofs, it will quickly fill up. They're no denying AI is making strides, but pinning it to memory is an oversimplification.

I don't have to open the article to be confident it's not worth reading. Anybody knowledgeable in the field should be familiar with AI writing tells and the message they send. It only takes a few seconds thought to transform the title into something like "AI beats mathematicians by out-remembering, not out-thinking." Regardless of whether the article is slop or not, I expect any competent writer to avoid slop phrasing in their titles. To do otherwise signals laziness.