Hacker News

What does this mean?

There are 1217 problems that Erdos proposed, 595 of which are open: https://github.com/teorth/erdosproblems

Unlike traditional benchmarks, it's difficult to overfit your models to produce flattering results to unsolved problems. The open problems very likely do not have published solutions (the initial batches were merely models surfacing data that wasn't published in obvious places, but we're past that now). New models will exhibit something novel by adding solutions. And the matter of solution is interesting as well (contradiction versus a positive proof).

I would venture to predict it'll take years to decades to get to 0 open problems. But I'd be very happy to have this comment look foolish in retrospect as models continue to improve

IQ tests are incredibly good at what they're designed for, which is discriminating relatively higher intelligence humans from lower intelligence humans. Also discriminating within a single human - they are routinely and reliably used to track cognitive decline.

For these purposes they are highly reliable (repeatable, internally consistent) and valid (correlate with ~everything to about the degree one would reasonably expect).

They were never designed for machines or non-human animals.

Nor were they designed for rare ranges of intelligence - these are by definition hard to create tests for, since it's hard to gather the sample sizes you need. So they work well for the middle ~98% of humans but can't discriminate well among the most profoundly intellectually disabled nor among true geniuses.

When I was 18, my high school girlfriend took me to the local Mensa chapter’s New Year’s party because her mother was a member and she was used to hanging out there.

It was a useful lesson that whatever IQ tests measure, it is completely devoid of value or interest to me.

Pure software benchmarks might be getting saturated, but physical ones aren't.

Let LLM control a physical robot to perform tasks that average human can do.

Nit: Turing’s actual imitation game is a party game (like Werewolf/Mafia) and nobody’s even trying to win at that. The LLM’s will just tell you they’re an AI.

I don't think IQ is a good measure for intelligence at all. Neither dolphins or octopuses can solve IQ tests.

Why? Seems like benchmarks that closely mirror the tasks you'd want an LLM to help with would be a lot more useful than some general intelligence benchmark.

Give away access to the model and go ask people from time to time if the model was of use to the person and if they were able to make the model work with them.

I am no where close to qualified to do that. Hell, experts can't even define what intelligence is, much less define a test for it

> basic inferential logic puzzles spells out

It spells out a form of intelligence - some can and some cannot.

Those puzzles are an abstraction of a skill which is thought to be exportable in other domains.

You forget the benchmark. The human subjects were told they were being timed. If you believe the lowest time is the primary metric you will absolutely trial and error at speed instead of meticulously plan out your moves to minimize that metric.

LLMs are not timed and given that it costs tens of thousands of dollars to run this test they're not optimizing for speed.

So you've got a deceptive test, with one metric being told to humans and not applied to LLM and a hidden metric humans aren't aware of but LLMs are as the test.

This is flawed from the get go. It almost seems like this was deliberately setup to be able to claim AGI and superiority of LLMs

Doesn't "saturated" mean that essentially there won't be any more progress in the benchmarch? Also of note is that two of your points only mean something on an occidental capitalist system.

these just need more compute:

- earn $100: easy to verify, hard to learn

- increase paid subscriptions in an A/B test: easy to verify, hard to learn

but we both know these examples go against the spirit of my point

theres no budget constraints for AGI

Nope that will be commodified in short order.

capability to match or surpass human intelligence across all conceivable cognitive tasks

What human intelligence do you mean? Genius? Professional? Educated? Random person? “Dumb” person?

So vague it’s useless.

We cannot in a declarative sense define what is economically valuable work even now let alone into the future.

People take what they can get for pay. Very few individuals can demand a wage. The value of employment is obviously designed around that, not some arbitrary definition of “valuable”.

Of course an AI will accept $0/hr, it doesn’t mean it does the job.

Anyone who could accurately define the value of work would be wildly successful without having to try.

That is not a useful definition for me unfortunately.

Right, you are interested in “AGI” and presuming none of that requires consciousness right?

For example, how do you know that “feeling pain” is not a functional prerequisite for a task. And that consciousness is a prerequisite for feeling pain

Unproven, but it could very well be some problems require consciousness to be solved.

So basically, being able to set it free on some long running goal and it sort of “lives” and autonomously does its own tasks?

I wonder at what point consciousness is necessary… that is, if you can have anything like that without it.

To the point that solving consciousness (and combining it with intelligence) is what gives you the autonomous, recursive, self-improving thing otherwise it can only drive in the dark and make big mistakes.

To your point I think - it’s why we don’t see too many non-conscious advanced biology (it rarely survives against those with it).

Didn’t I just see a thing about how actual unemployment is at like 24% a few days ago?

AFAICT nvda's result is on the 25 open problems, while this submission is on the "semi-private" set, ran by the arc people themselves.

I rest my case.

I agree, like the average human isn't generally intelligent.

IMO, AGI is literally no different from ASI, though people think it is. Like, Imagine you have 1,000 generally intelligent humans working for you (which nobody is really) and you were to point them at your pet project. That would be amazing!

Hilariously it is, I'm just reading more on this!

So, you’re telling me I need to start a benchmark as a side gig to get a bunch of free compute.

Astra please create a benchmark that’s favorable to your reasoning skills with a human interface but don’t make the score too attainable add some small issues that keep you below 100% to look sensible and to keep my evaluation metric side gig going.

Alignment++

It’s not that different than a lot of real world economies. Often paying for someone or something with better quality can reduce total costs. You have less failures, less mistakes, so on, so while the expertise or quality of the product is higher than cheaper solutions, they can be more reliable and over time ultimately cheaper.

The question I have is how far back that curve can go without relying on economies of scale to just drag all the points back to the left. And without overfitting a specific metric that I don’t need (like this test).

What is the intuition. Higher quality turns due to more reasoning results in significantly fewer turns taken?