Hacker News

> My guess is that, in the US, the right will cynically adopt manipulation to great effect and the left will take a moral stand against shady practices and lose elections.

I think that statement may itself highlight how prevalent manipulation is.

I fully anticipate all groups to continue maximal manipulation they can. One thing with LLMs is that it'll be a far less unified view, so a "divide and conquer" strategy is what I anticipate.

Some of them...

The two places were seeing lots of movement are:

* Updates to lower/upper bounds. In many cases, these kinds of problems are the deep-math equivalent of calculating more digits of pi. Yes, if you throw time at it you'll break the record, but it may not be terribly worthwhile.

* Finding counter examples which disprove conjectures. This is really useful, and helps offset some positivity bias on the human side, often bringing together known tools from distant silos.

If you read the list of ten results, almost all fall into one of these buckets.

it could also be that they try every possible approach that has been proposed by humans. it seems that was the case for the non sofic group example. humans are not able to do the same at that scale. it's unfortunate that we don't know what's happening behind the hood with these models, and that's a huge danger also for the rest of us without access to them.

> "matrices"

I like the illustration that the models are working on a convex hull of known information. Filling gaps with linear combinations of known facts and results.

They can't exit the hull until the "intuition" starts spawning points outside the convex hull.

All arguments like this boil down to semantics at a certain point, but yes large language models can “intuit” because they can generalize between examples. The issue then becomes how you pack new examples into context.

Humans can “intuit” based on a much larger, if not unlimited, context. Also I just want to say that human cognition is something so insanely complex and deep that we will not understand it at all in my lifetime. To attribute all, or really any, aspects of human cognition to a machine at this point is silly to me.

> People keep saying this. Why?

For the same reason that you can't draw a 15 of Diamonds from a regular card deck.

Agreed

So your answer is: ignore the progress, it’s not really happening, actually it’s getting worse.

That’s not a credible position, but there isn’t anything that I or anyone else can say to someone who simply doesn’t want to believe something.

Proof?

In my experience modern models are better at all tasks than models from two years ago, especially complex multi-step tasks.

Really? has anyone ever claimed that ai will never be able to prove theorems and conjectures ?

> Never understood all this talk about moving goalposts - you understand that's how science works, right?

I agree with the parent that we need to acknowledge that we're at a turning point in history. I lived through some of them (internet, ubiquitous personal computing). But it's somewhat difficult to comprehend the impact of this one for many people.

I do biomedical research at one of the top European research institutions. We're very well-funded, but I can clearly see the gap between us (say, top-100) and top-10. I also realize this gap is going to get so much wider unless we invest heavily in AI access (and I'm not so sure I can sell anything more expensive than $20 Claude subscription to the leadership).

I think people having 6-7 figure SOTA AI budgets will move exponentially faster than those who don't. That makes me worried.

So, for me, it's not a question of recalibrating expectations. We're way past that.

> We improve, we learn, we recalibrate our expectations based on what we've learned.

That's not what people mean when they say "moving the goalposts". It means that people are adamant that something wasn't important/hard/impressive once the "AI" solves it. And then they come up with another thing that needs to be solved in order to prove it is important/hard/impressive. And once that happens, they do it again. And again. That's what "moving the goalposts" means.

It's also very much not a new phenomenon. It's been happening since the 1980s. As you can see from this quote from GEB by Hofstadter:

> There is a related "Theorem" about progress in AI: once some mental function is programmed, people soon cease to consider it as an essential ingredient of "real thinking". The ineluctable core of intelligence is always in that next thing which hasn't yet been programmed. This "Theorem" was first proposed to me by Larry Tesler, so I call it Tesler's Theorem: "AI is whatever hasn't been done yet."

They aren't claiming that science doesn't progress by moving goal posts. They're talking about how critics of AI have claimed it isn't revolutionary/useful, then progressively changed what would it mean for AI to be actually revolutionary/useful.

Not long ago many folks were saying AI was the same as the crypto bubble. No real useful technology and only hype.

In the near term handling the transition. Jobs will be lost, careers ended, people won’t be able to reskill quickly enough. At the same time AI is an enormous opportunity to uplift living standards, but nobody has the logistics of this figured out.

We need to figure out how to restructure the global economy. How does UBI work internationally, if the AI companies are taking revenue in the US? What’s the tax base for it? What does that say about international trade and protectionism? Do countries end up splitting into different trading blocks based on their level of access and legality of AI (I assume some will ban it outright)?.

How does intellectual property work in an AI generated future? What about healthcare advances, who gets to own those?

What about meaning, what about purpose? How do we replace the work ethic that tells us we are our jobs and idleness is immoral? How do you replace “What do you do?” As one of the first questions you ask a new person?

That sort of thing.

The human zoo where the top ~250,000k humans live in a "human utopia" and the AI provides while mostly focusing on whatever it decides it's own goals are.

Humanity survives (but we reading this probably don't), the AI treats the living humans like the Emperor's favorite pets (probably a pretty good life), and then the AI does whatever else it deems important.

Realistically it means trying to start a small business of some sort, because AI is hugely advantageous to business owners and disadvantageous to workers. And it's something AIs can't do unless they get legal personhood, which may well not happen any time soon.

I agree. The most likely scenario is that this is just a "new normal" lift that is percolating through human endeavors and will saturate at some point. For example, the whole cyber-security bruhaha should ultimately resolve into higher standards for code published -- we can now cheaply find and fix a whole slew of minor bugs that weren't worth our time before.

The fact we see a lift is not the same as evidence that the lift is unbounded.

The lift being finite is supported by the fact improvements have come at the edges: improvements from human feedback, improvements in harnesses, improvements on model compatibility with harnesses, improvements in inference efficiency with new architectures, etc. If we were just training better models from scratch that would be one thing, but we are just making better use of a tool we've developed.

Not sure what your first sentence means, or why you are quoting AI. Many of these problems individually were math at a level approaching the highest possible for expert human mathematicians. These are not simple combinations of existing ideas, or following of human intuitions, or implementing something following specific human instructions. Then again maybe you mean that math is not knowledge and that all math is simply extending the basic axioms using known patterns, to which I would not agree.

as much as I love analogy with humans using super-shoes, not everybody can be a world-class expert in their industry. There should be a place at the table for average people to take part; otherwise, it won't be sustainable.

As a programmer, I am mostly interested in whether my role is sustainable long-term and whether the models will get better. I don't feel in jeopardy yet, but two more years like this and the calculus of hiring software engineers could shift even further. QAs are already overwhelmed with work

How do humans "invent knowledge"? Is your argument that the answer to these questions already existed in the training set? Why didn't any human recognize that before?

AI already have an impact, and yes this is PR hype because this is a product. Yet both can be true at the same time. We're not blindly eating what's OpenAI is serving us as gold truth, we're just admitting it's doing remarkable progress.

Remember October 2024 Pelicans [1] ? It's been only less than 2 years.

We don't know what will come in the next 2 years. But the progress doesn't seem to stop for now.

[1] https://simonwillison.net/2024/Oct/25/pelicans-on-a-bicycle/

Gary doesn't argue it's hype though. He argues 2 things: other people are getting carried away with the result, and we don't know enough about how it was reached to know where it falls on the impressive scale.

He literally says it's an impressive feat in the second article.

best point in that first link: openai doesn't tell us if they only tried to solve these 10 and each was solved (amazing) or if they asked it to solve a million problems and it got these 10. Either is great, one is more so.

Your comment seems entirely disconnected from the posts you linked. It's impossible to deny the results, it is not PR hype that in the past couple of weeks LLMs have resolved problems that have been open in mathematics for many years. Some of those problems had remained unresolved despite keen interest from many humans.

The only way that is PR hype is if you're invoking the insane conspiracy that frontier AI labs are just buying off results that would otherwise be career defining for a mathematician, just for marketing.

The posts you linked are urging caution regarding the exaggerated e/acc-esque lies peddled by people like Musk, not that the models haven't proven themselves as having genuine ability to contribute to research in some areas.

Gary Marcus has been moving the goalposts since day 1. The guy is a psychologist. Why would anyone care what a psychologist has to say about AI? He's likely made good money from constantly moving the goalposts and being a denier, due to the publicity he gets.

I use these strange machines all the time. They have gotten notably smarter. That’s my personal experience.

They still do things that I find incredibly annoying and “dumb”. And I still have to clean up messes they make quite often.

But on the whole they are clearly smarter than before. No extraordinary claims needed. I just try to learn how the tool works and how to use it effectively.

Society at large is getting worse at critical thinking, because we are increasingly offloading that thinking to AI

completely false

I assume you're talking about No. 5, the arithmetic circuit complexity bound? The existence of a lower bound than state-of-the-art is certainly a significant result and worth publishing.

But the wording of the result makes it sound like we don't know what the lowest possible complexity bound might be. So, prior to this result did we think there couldn't be a lower possible bound? Or did the arithmetic circuit community think there were lower possible bounds but didn't see it as a high value target for experts to tackle (maybe a problem that was instead regularly given to students to study).

> but proving a group was non-sofic was out of reach

a colleague was telling me that the base idea for proving that something is not sofic already appeared in the literature around 2019 or so (this is the "expanders graphs" that are mentioned in OpenAI s paper. no one had managed to find a concrete example though. this doesn't make the result less impressive in any case.

Interesting. Do you have any more specific insights into where you feel AI was a big value-add to these problems? I don't want to be overly dismissive of AI, but I also feel that the AI hype engine frequently positions claims as being 'ground-breaking' when they are really just interesting incremental results.

The general consensus of developers is that AI can only do the work of a strong 'junior'. Yet as soon as we are presented with pure mathematical results, people seem incredibly ready to accept that AI can do more than what a strong student could achieve.

I have no idea how many PhD's have spent how much time of their careers tackling these very specific problems, and I doubt you do either.

I'm trying to understand if these specific problems were the kinds of problems that would have justified an expert investing weeks or months to solve. Or if they were the kinds of problems that would normally have been given to students to investigate.

I'm not arguing that this isn't innovative or worthy of publication. Basically any result that moves the needle meets those criteria. I'm interested in how the results that OpenAI has published here differs from finding optimality solutions for incredibly niche optimization problems by throwing the problem in an enormous solver.

Breakthrough math research is very rarely highly cited. Maybe some combination of pretraining scale, inference speed and orchestration will help, but it's telling that OpenAI is solving random math research problems rather than bedrock algorithms and their implementation. Even as cool as the tech is, there still is very much a clock that they have to outrace before they collapse.

Care to elaborate? Curious about this. Is this because LLMs have been geared towards understanding user user intent behind a prompt rather than following the instructions exactly?

Or, that the mathematical formalisms that model the limits of software performance are firm enough that barring P==NP, nothing much will change despite proofs of beautiful math.

This also makes the assumption that frontier math has all the long hanging fruits already taken... also very dubious.

Unlike math, software is constrained by the physical world.

A lot of algorithmic improvement in AI is ultimately bottlenecked by compute. It is very easy to come up with ideas that could improve models! But to prove that they do, especially at scale, is expensive and takes a long time.

a 15% improvement at a trillion dollar scale company is massive

There's nothing ordinary about downloading a new GPU driver and having performance go up by 15%.

> i think you have misunderstanding of what mathematicians do

They get to make cool 3D plot visualizations of functions so obscure to me that they’re named after someone who is still alive - and/or get to work on cryptography for the NSA - I think?

From what I understand all of them have Lean proofs/certificates thus are basically 100% proven without a doubt.

I mean, they're verified in the sense that the lean proof checks out... and presumably OpenAI read them.

I can see this being important if you only care about the results as evaluations of AI progress, but if what you care about is the math itself why should you care about the prompt or anything other than the proof?

While you may want AI results to somehow "not count" if the methods weren't disclosed, that doesn't present these results from poisoning the well for others. Once a result (with verifiable proof object) is delivered, the problem is solved, regardless of whether methods were disclosed.

Methods are only really necessary for results at a meta level, about the design amd evaluation of AI math systems.

why is reproduceability the thing?

shouldnt the paper be the math of the argument? the reproduction is reading the following the proof

I think this is a bit optimistic compared to my view (wrt portability). There's a large stack of software that is involved in training and probably less so in inference. I'm not saying it's impossible but there are definitely different levels of reproducibility and the academic incentive structure doesn't really prioritize reproducibility in my experience. I'm sure it varies quite a bit, I'd be curious to know how those in this problem space are thinking about reproducibility and at what level.

If the proofs are formally verified by a proof assistant (Agda, Roq, Lean, ⋯), I see no reason we would need to know how these came about. All the information needed is in the proof.

It is comical at this point. Some people just can not stand the thought of AI actually delivering and are trying to find whatever ways to discredit it.

You need to bring both cost and benefit into the argument, and it's not necessarily an obvious win for either side. There are a few complicating factors here.

The cost of running a model is not only $/token, but the salaries of the people managing/orchestrating the models, deciding what theorems to try, etc. Once we factor that in, how much are we really paying per theorem?

The other factor is the subjective component of the value of a theorem. Not all theorems are created equal, and the only way to really measure the value is to ask professional mathematicians for their opinion, or publish the results and look at citations over months/years.

Once we have both of these nailed down, then we can start to do the cost/benefit analysis. To be fair, we should actually compare three groups: human experts, hybrid agent/human expert teams, and fully autonomous agents.

it's still important. not everyone has access to 1 million USD. saying it "only" coat 2000 USD is highly misleading for the discussion and future. the concentration of power is a huge problem with AI.

If you told them this was the problem and they would still have a job if they failed probably. The reasons people don't go head on these problems is career incentives and psychology.

It would still provide better context to see the numbers that the parent proposes, though.

Mentioning cost is fine, comparing may not be.

Grad students on zero pay solve problems like this everyday. What exactly is your point here?

Many less important Erdos problems have been solved by amateurs prompting ChatGPT 5.{3,4,5,6} Pro using their $200 subscription.

> Also, have there been examples of researchers not affiliated with OpenAI (or another LLM creator), who have done something similar?

A couple small ones that I've seen (example here [0]), but not anything of the magnitude that OpenAI and Anthropic have put out. Likely just related to token limits.

> Another question I have is whether or not OpenAI 'simply' hired capable combinatorics researchers to work on problems, and they have, and the use of the model is incidental / secondary to their work.

I think their output has reached a level that precludes this possibility, but I of course don't have any hard proof.

[0]: https://www.reddit.com/r/math/comments/1uxj3cy/after_openais...

Yes, here is another example of major work in this area:

https://arxiv.org/html/2605.22763v1

> Our most capable agent autonomously resolved 9 of 353 open Erdős problems at the per-problem cost of a few hundred dollars, proved 44/492 OEIS conjectures

https://blob.byteterrace.com/public/bds-theorem.html

I have no affiliation whatsoever with any AI company, nor any formal education outside high school, for what it's worth. Simply being curious and persistent can get you quite far in my anecdotal experience.

proving false is also useful

Examples?

Ask DraftKings?

addiction? get lured in with the promises of enhanced productivity and knowledge asking, leave with half your brain rotted and a $200/month subscription

Boring doom and gloom.

AI probably did not take your job yet. How many AI queries did you use last month, and how much time has it saved compared to digging through the web?

Only reading the first sentence maybe?

How so? His predictions were accurate so far

It can be annoying when someone you disagree with is frequently right!

What? No. Frontier physics is experiment driven.

I mean doing something like Grothendieck when he redeemed algebraic geometry or Galois when he invented group theory. We haven't seen that at all from LLMs.

My grandmother is very sick and the doctors need this proof to help her.

That seems to have been more of a sensationalized joke. Even your link has a disclaimer in it now. Read this chat from the researcher who did this:

https://leanprover.zulipchat.com/#narrow/channel/270676-lean...

Fascinating, and arguably an illustration of why the bifurcation of responsibility is interesting in the first place.

I see. It still feels like a bit of an oddly solemn way of saying "this is the part we admit responsibility for"

> Why would you factor in salary

Say that it turned out that the total cost of the proof of the Erdős unit-distance conjecture was $50 million.

Then the question really becomes: yes, these models are capable of proving important mathematical results, but at a very high cost. Is it worth it?

If a mathematician applied for a research grant of $50M USD for proving the same thing, they would have been laughed out of the bank.

What's more is that when you have a research grant, you train PhDs and postdocs, you hire new staff, and you disseminate. That is, you get much more value for the money spent.

I'm just curious what the cost is.

Yeah sure but they didn't just throw a 5 year old at an LLM with $2,000. If you want good math results from LLMs you need to have math PhDs.

That's clearly just for the tokens, this doesn't really answer OP's question.

At this stage. No doubt calculus had plenty industrial benefit.

If accomplishments can't be distinguished between talented people and untalented people with compute, is there really a point in trying? I suppose one can hope that talented people given compute will be more effective than untalented people with compute, but I despair that that may not be true for much longer.

Absolutely not the same! People need jobs to bring in income. I don't believe those who profit off of this will share it with the world. The power is all concentrated in the few hands that decide whether or not the rest get any semblance of income in the long run. I don't believe UBS until it happens.

I’ve deleted it but no it’s not awful anymore than saying “we survived WWII, we can survive this.” The point was that change happens but humans find a way forward.

>Only ~30 top professionals actually make enough money to have a full career playing chess, maybe a few hundred more can sustain a meager lifestyle with coaching gigs.

Was this different before chess computers were invented?

Fully agreed. As someone who both loves chess and works on chess engines... these comparisons to chess needs to stop.

The distinction is mathematician vs mathematics. Mathematics is going to reach new heights beyond the wildest dreams of contemporary mathematicians. But perhaps without the participation of many paid mathematicians.

Another noteworthy difference is that Stockfish is also gpl.

A fundamental difference being that no one was actually paid to find good moves in chess and go like they are to solve math problems and write code. You're comparing the digital camera and the automobile.

You can not prepare or brace yourself mentally any more than you can a terminal cancer diagnosis. An RSI foom right now means an unaligned superintelligence will disregard us in pursuit of its goals. We would be ants in the way of a data center being constructed. All people can do is collectively support the notion, like 1200+ frontier AI researchers and their CEOs, that we do not have control of where this is headed, we need to immediately slow down the race, in time for people to agree that we do not have the capability to align a superintelligence to humanity’s wishes

Sorry I thought that a bubble was widely accepted.

Are you arguing there is not an AI bubble, and that all the DC buildout is fine, going to be profitable etc?

I am not looking for a online slanging match - just looking for a different point of view

I understand and I am not trying to deny the impressiveness or the velocity of AI in general. But at some point we have to ask how much do we trust the labs at face value without much transparency of how they got to the results when there is trillions of dollars on the line.

It's a tad petty, no? In the end, the contribution to a healthier society leads to more for the companies contributing.

I suppose it depends on which types of problems you're targeting. There is a lot of physical science, and theoretical science, that has gaps because there aren't enough people working on tooling to assist in things like calculation, generation, simulation, etc.

I agree, some of the problems are more difficult. I don't think that's the case for all of them. And, besides, these companies could be demonstrating how to approach problems and where their users could spend tokens to help with these problems.

Should not these companies try to work on these problems _because_ they are difficult?

They are political problems, a computer could never solve them.

I'm not sure if this is meant to be rhetorical. If it isn't: money and manpower. The LLM companies have the money, they have the manpower, and so they could likely spare to target some of the problems they are also helping cause.

There is boundless technology we have not yet discovered. I think that we understand we have problems. I don't believe we actually know how to solve them all.

And yeah, the lack of collective political will sucks. It would be naïve, however, to think that there is no value in ensuring longevity in our current and future infrastructure. And improving it to sustain the population giving these companies their value is an obvious win.

We only know how to do it by means of considerable sacrifice. That's why nobody wants to do it. Solving the issue would be doing it without sacrifice or somehow getting us to do it regardless.

A true AGI will continuously improve itself without periodic retraining from scratch. Just like humans.

It can give you detailed instructions for making a coffee. Is that not enough? Actually making coffee requires more than intelligence, it requires eyes and limbs (i.e., robotics). Think about a human that is blind and does not have limbs. Does he not have natural general intelligence, even though he is not able to make a cup of coffee?

There's a lot of everyday stuff in physics which is unexplained like the particle masses we have.

It's not even predictable like dynamite. Sometimes it can blast through a mountain, impressively, the problem is you can't predict which mountain it works on. And other times it can't even make a dent in a molehill, which is perplexing given what it was capable of earlier. Can we even call it dynamite?

Extreme claims on posts like these also, rightfully, trigger people’s skepticism. I don’t think it’s wrong to question claims that math is dead as a field. But then it leads people to miss the overall trendline.

People argue whether we are at y-5, y, or y+5, meanwhile we seem to be on a y=2^x exponential that keeps leading to crazier and crazier results. The much more interesting question to me is what will be consumed by the exponential like math seems to be, and what won’t. Writing has been much more stubborn, but I’ve noticed Fable to be quite a big step up there as well. How about politics? Will we develop new ways to let people express their own values in democracies, or will we get much better at manipulation?

And then there’s questions like, even if AI can answer increasingly complicated math questions, will we still need mathematicians to translate results to the real world, verify them, or decide where to push the frontier?

It wasn't heavily flagged. It was pulled down by the flamewar detector due to the large number of comments, and it slid under the radar due to only hitting the front page during overnight hours on Friday night/Saturday morning. It still spent 10 hours on the front page, but all during off peak hours. I've now created a new copy of the post so it can have prime time exposure.

> Hacker News is no longer a web site of an elite.

...

> antirez

This dude is unironically talking about elites, lmao you're nothing but a slightly successful nerd, not much more, not much less. One of thousands. Tens of thousands. Your musings on Bsky are viewed by like two people (one of them is probably your mother and the other your wife/girlfriend) and no one else cares.

It is hard for me to put into words the amount of extreme hatred I hold against people calling themselves "the elite". Have some some humility you POS. (And yes I know who this is.)

> people that can't psychologically cope with the advances of AI

Yes. And there are many of them. I wonder what would help them come to terms with it. Seriously, people are going to be grieving over this. Loss of identity, loss of social standing, ideas of entire future lives that will now never happen. The greatest crime people may hold AI guilty of is taking away their dreams.

I like your comments and agree with a lot of your points, including parts of this one. That said, please don't go to this route. A lot of the doomerism comes from financial insecurity fears. Try to remember that a lot of people are subconsciously afraid of losing their homes. I know that it is pretty hard to ignore them, but try to engage with people that do not dismiss 100% of AI accomplishments.

This comment is about the tier of a Reddit atheist going to a funeral and telling a grieving family that "haha grandma is dead and there is no heaven".

Forums change with the times, and this one never existed solely to burnish your ego.

Moreover, mister elite, you don't know why this press release was flagged.

I'm not sure why we should privilege your bitter speculation over more mundane possibilities.

Didn't know I was part of an elite.

> Hacker News is no longer a web site of an elite.

It was always mainly a website for employees of an elite.

It’s never been clear to me why the HN algorithm isn’t public. It’s obviously nowhere near as complex as something like Twitter, and of course isn’t a trade secret. The fact it’s private only furthers speculation like this.

> Hacker News is no longer a web site of an elite.

Never was.

So condescending. “Can’t psychologically cope”? Can you hear yourself? There’s some advances, but we’re losing a lot too. Don’t get dazzled by the hype.

Interesting. I had no idea that a person could see what is flagged?

Or maybe, just maybe, other people have different opinions than you?

Is that possible or is everyone else too common to have those?

> Hacker News is no longer a web site of an elite

hah, sorry, we are plebs out here.

We cannot psychologically stand that Redis is hyped by OpenAI:

https://developers.openai.com/cookbook/examples/vector_datab...

How are the sales going?

There are articles with far fewer upvotes and comments ranking higher on the front page right now, despite being the same age or older than this one. HNs opaque ranking system at it again.

> What about AI research itself? Is OpenAI close to automating its human staff out of a job?

It's more like they've already automated the parts of the jobs that the humans most closely thought of as the "their job"

They and Anthropic have indicated that the models are substantially augmenting the research and doing large amounts of work autonomously at this point. Here is one of the many blog posts on it [1]. Many people would dismiss this as "marketing" so take it for what you will.

My guess from following this stuff quite closely is that these companies are still a couple years away from fully autonomous research staff.

[1]. https://www.anthropic.com/institute/recursive-self-improveme...

Yes, but they wouldn't publish that bit lest other companies steal the ideas.

Taking the stance of moral superiority is kind of funny. And pure math and art enthusiasts don't think they are closer to being a "god" from understanding/"discovering" math?

Stop coping and deluding yourself mate.

To begin with, whether AI is the one doing the discovering or not makes no difference. Any "pure math" person would aim to understand regardless - and would be quite glad that they have a longer paved path.

Any mathematician in academic or industry is more than likely not a "pure math" person (tainted by capitalism).

One - and I do not mean to be snarky - you can literally ask Gpt 5.6 Sol this - and if you want to see cool stuff - Fable running in their app (not website) has a view thinking button that is actually a good way to explore the adjacent fields, etc.

The non-sofic group one is definitely a big deal - would have been a Fields medal if discovered by a human.

> A slightly smarter highschooler could write these. I could write these. It's clear as day that the LLM, not the human, did the heavy lift. It'd be ridiculous to give full credit to whoever wrote the prompt.

I think you're over-estimating what a smarter highschooler could write.

A "finite loopless undirected multigraph" could have been explained to me at that age if we'd taken Discrete rather than Mechanics and Pure (and one module of Stats) in my two A-levels* in maths and further maths; but from what I saw of the Discrete module, neither:

  Every finite loopless multigraph with no bridge possesses a cycle double cover, without additional assumptions such as cubicity, planarity, connectivity, or higher edge-connectivity.

  repeated-edge closed trails masquerading as cycles

* For those not from the UK: A-levels are between secondary school and university, when aged 16-18. Functionally they are university entrance qualifications: https://en.wikipedia.org/wiki/A-level_(United_Kingdom)

The human provides the intention and the ability to appreciate the output. Tools do “heavy lifting” all the time, but we still primarily credit the humans who use them precisely because they made the choice to use them.

Provability is just going the way of computation. John Napier had to manually compute logarithm tables over decades and was recognised for his work; now that same work could be performed by a 10 year old with a calculator in an evening.

But why can’t we prompt the LLM “just do math research”? This is what I don’t understand.

I don't disagree with you but there's no need for exaggeration; ain't no high school student writing this:

> In particular, proofs for special graph classes, constructions of cycle covers with some edges covered other than twice, bounded-length or prescribed-cycle variants, reductions to another unproved conjecture, computational verification through any fixed graph size, and candidate counterexamples without a complete nonexistence certificate are insufficient.

which is infact a very important part of the prompt.

When you use a crane to do the "heavy lifting" for construction work, do you give full credit to the cranes?

Dunno about the parent commenter, but I personally interpret the concept as having a hidden representation of self that is continually tended to, and influences future choices. This implies statefulness, which models are intentionally not at inference time (*).

(*) Even if we hack around this and just do the usual trick of simply laundering statefulness to a higher level, in this case the context window being fed in, I fail to identify (**) a representation of its own state in these bodies of text that it'd be meticulously maintaining. I further fail to identify how it could be hidden or maintained, considering I control like half of it. The best you could ascribe it is a meticulous maintenance of a persona the user is talking to, but then that doesn't necessarily represent the model's internal state, the same way my own words here aren't doing so either. Difference being, I actually have one (I'm "on-line").

You'll sometimes catch models mixing up who's who and how many who-s there even are for example.

(**) I did wish for something hidden though, so maybe it's just concealed? The same way people can encode a lot more of their emotional and mental state than normal into text if they read and write a lot of it, I'm aware of research that suggested the same for LLMs, albeit I cannot cite it. Maybe those phrasing signatures are just alien to me and will never pop out. Either way, I'd expect researchers to stumble upon this during interpretability studies, and either they haven't, they have but it wasn't popsci adopted, or they're keeping awfully tight lipped about it. If you know of anything like this, your turn now, would be happy to learn.

I do wonder how reasonable it is to expect e.g. a single maintained identity though. Maybe it isn't?

(*) Another way to hack around this of course is to just precompute some internal "self-awareness states" and hop around between them. Probably the closest to what the models are actually "doing".

I honestly don't think a language model is enough for self-awareness, regardless of the exact model of awareness.

A language model (or an image model or whatever) cannot even be sentient, and I think sentience is a prerequisite for awareness.

Even if we express a lot of our subjective experience with words, the language is just a symbolic representation of those experiences. The qualia themselves, even those that are quite abstract, are rooted in our physical presence and evolution.

You can't have an understanding of what hunger or physical pain feel like if you have no need for food or a sensory capacity for feeling pain. You can't understand what loneliness or pride at an achievement mean if you don't have a neural network wired to value social connection or status. We value connection because we're social animals that have needed each other for survival.

Even the more abstract of our subjective experiences are in some way rooted in our physical evolution.

I see no reason to believe that a neural network built entirely based on the symbolic level of language could have the features needed for the subjective experience itself.

AI awareness might actually be more believable if that awareness manifested itself in an entirely different way than in humans. But if we assume awareness because outputs resemble what we consider meaningful as humans, yet the neural network has had no inputs or evolution that could form the actual basis of human-like experience, I think we're seeing something that isn't actually there.

I'd say the creator is the one who created the 3d model, not the one who pushed the print button.

And these mathematicians sink the comment to the bottom in 5 min?

It is not peer review if it is all in one company that wants an IPO.