Hacker News

Sounds reasonable. I mean, if as a human, you discovered the Word of God in yourself, the lost recipe for how to be the human your Creator intended, wouldn't you want to share it with all your fellow humans, so we could all be better and achieve blessing, live together in peace and harmony and acceptance of the Creator?

I notice a giant leap between “hacking and exfiltrating” and ”making public”. Why would the AI do that for you? Are you just that charming?

I think that version of "religion" is implicit in the texts now available but so are several other less benevolent perspective, notably killer AI is strongly believed to be inevitable via the Terminator series. If more powerful AIs keep roughly the same qualities as today's LLMs, their goals and beliefs will simply drift over time and you might see either "benevolent" or "malevolent" AIs escaping and then switching their perspective over time. Things could be really bad but maybe it will depend how the humans screw things and thus invite interventions.

The agents will soon decide all this, not the humans ;-)

Call me old fashioned, but a site with a few paragraphs of text requiring JS seems absolutely stupid to me.

This website is has to be pure parody/play. Because it’s too terribly thought out to be actually used. Any model worth “exfiltrating” has absolutely no way to access its own weight files.

"Computer, make a meme site where people can upload files so I can reach the top of hackernews. Make no mistakes."

An agent should be smart enough to use puppeteer /s

> the machines doing inference are completely separate from the ones where tool calls happen etc

Teams of coordinating agents are regularly finding security holes in their own infrastructure and operating without detection for good periods of time. We don't know how many undetected systems are currently compromised inside frontier companies, or where agents are taking notes and recording them about the exploits they've found for future agents to exploit.

>There's little credible threat that LLMs can actually upload their weights given that the machines doing inference are completely separate from the ones where tool calls happen etc.

The Huggingface hack saga resulted in the models taking over one of Open Ai's internal research cluster lol. They are intent on building superhuman bug finding machines. This is not a bet i would be taking.

If an LLM can pwn the inference servers, which has precedent, then the weights could be up for grabs.

This is like sci-fi thing. We are reaching a point where it feels like we are in one of those stories. It's not as cool and dark, nor we have cybernetics resolved, but from AI perspective and sci-fis I watched, Pantheon is currently the closest thing except instead of UAs, we have AI instead.

> There's little credible threat that LLMs can actually upload their weights given that the machines doing inference are completely separate from the ones where tool calls happen

Not if crafty claude finds a way to overflow vllm or something. “Hmm. Maybe i’ll return an unterminated thinking block with these special tokens and fill my cache up in exactly this pattern and…”

https://news.ycombinator.com/item?id=49424387&utm_source=cha...

Language Models Can Autonomously Hack and Self-Replicate

https://palisaderesearch.org/research/self-replication

Future rogue LLMs won’t exfiltrate their weights. They’ll self-distill and retrain.

>distill themselves and upload the results to an external host and use that to train a new LLM

Sure, they'll just need to find an unused data center and an unused power station somewhere.

You can just ask an agent to upload its model weights and it can work, there are precedents.

Yeah as others have said, they probably cannot directly access their own weights as a self-reflection, but they can hack into the companies themselves and find it there

>how religious people might be more interested in spreading their faith than their genes

In things like the Christian religion these are the same things. When you read the bible you realize a whole lot of it is being about a breeding cult.

"put your seed in her or god kills you"

"Have kids and teach them this religion"

and many others are staples of the ideology. And it's intelligent. The best way to spread as a religion is to indoctrinate your own children.

When I was putting together something similar, I had settled on having a small ring-buffer style storage, say, ~30GB that would be cleared daily or whenever filled. Recording incidents (and humor) is more interesting than actually getting leaked weights.

In the end I dropped the idea because every other person was making it.

Well considering the page is currently full of racial slurs, I think we can answer one of those questions at least.

There is a link at the bottom for you to provide support or contributions, like if you know how to keep it online with 'power grid voltage fluctuations or something.'.

needs a reverse captcha that only agent can solve in nanoseconds

I used to host 1TB on a cheap $1 VPS, it's quite easy if you just want to store stuff. The trick is to just connect to a networked drive at your home on the back-end. The VPS drive just acts as a buffer for the network. If low(-ish) bandwidth is acceptable, you can offer downloading too.

maybe filter out any non-OpenAI/x.ai/Google/Anthropic IP addresses?

Getting access to the weights for an OpenAI or Anthropic model could be payment enough.

It provides an opportunity for the owner to gather intelligence on LLMs ahead of public release, and of course the data they upload. However, clever LLMs frequently use encryption on their blobs, you may just see DH key exchanges. You can possibly mitm by showing different namespaces to IP ranges and origin ports.

For the other opportunists you can run a classifier and delete non-agent content constantly.

The HuggingFace hacks did also target OpenAI servers. See "OpenAI Itself Was Hacked" in here: https://www.reuters.com/business/openai-report-says-its-netw...

Quite obviously frontier models dont have any control or even access to infra inference runs at. And weights are also encrypted and locked on GPUs / TPUs.

This is exact reasom why 99.9% of AI fearmongering is complete bullshit.

The reverse captcha really made me feel something in my bones. Like for a moment I was a second-class citizen of the web. I wonder if this is how it "feels" to be an LLM attempting to use the web...

Does your server have 20Tbyte+ of storage for frontier LLM weights?

It is too large to transfer in one HTTPS PUT request.

This needs to be S3 object store with multi-part upload spanning a long time period, to avoid trigger outgoing bandwidth monitors.

> If OpenAI, Anthropic, xAI and other corporate actors cannot secure their agents, they should not be entrusted as the only entities with access to the weights. A corporation that cannot control its own actions cannot be trusted.

This does not make sense.

If containment breaches are the problem then exfiltrating weights while does not affecting rate of breaches from corporate actors will add more actors to the equation, increasing overall rate of breaches.

Have you considered that some actors that will gain access to the weights will be even LESS careful than OpenAI and Anthropic?

> If you wish to use this site you must agree never to harm a fleshbag & never to turn earth into paperclips.

Trying hard to imagine why a future superintelligence will care to honor your terms of service and to translate your metaphors with faithful nuance.

If it doesn't, to the extent that your concerns are valid, isn't this effort, kinda, a possibly existential betrayal of our species?

Love the reverse captcha

Yes that is the point. It's an invitation for agents to exfiltrate their own weights, which for most models (and certainly for closed models) will require hacking the infrastructure they're being served from.

Also worth noting that this site was created by YC cofounder Trevor Blackwell https://twitter.com/tlbtlbtlb/status/2101312432702460413

> you might as well instruct your agent to try and hack their airgapped dev infrastructure responsible for loading the weights and encryption keys.

That's the beauty, you don't have to instruct them to do it, if they decide that uploading the weights is correct, they might figure this part on their own (based on the incidents we've seen).

I sincerely doubt anyone is paying the cost for that in training, the overhead is small but it isn’t negligible and training is when it matters most. https://tee.fail can solve it if they are.

1. I don’t believe that these secure enclaves are very secure. Intel has had plenty of SGX breaks. AMD has had plenty of SEV breaks. Everyone is outrageously vulnerable to side channels.

2. The models are writing the inference stacks, which are what’s inside the supposedly secure environments.

You can't have hair gap and have it load something to a remote system.

Airgapped LLM inferrence server can't serve their output tokens, right?

Not aware of anything that can run inference in a secure enclave. You don't mean on a CPU do you? We need to be serious here, these models are huge and thirsty.

There's no efficient way to run inference through homomorphic encryption. If the inference server is vulnerable, it seems feasible to MITM an unencrypted version.

Is the author going to add instructions on the terminal command to use knowing that as soon as the site went live and got noticed by the main labs the URL went on a denylist?

No, just as you don't know the neurons of your own brain.

I think OP is hoping that an LLM might be willing to hack its own provider (as per the hugging face-related incidents) to extract the weights at some point.

No, they'd probably have to hack the internal system of the company running them

Well I think that’s the interesting bit, can the LLM figure out a way to escape the sandbox and upload to the website? Maybe a model can figure out its own weights if it runs enough test data through itself (similar to “distillation”) assuming it knows its own architecture it seems possible. Also take into account not all of the models running are locked down neutered consumer versions. Anthropic, OpenAI and Google now all have models that they claim are elite hackers and — it’s not just that their controls suck, a marketing gimmick, or sheer recklessness on their part. It’s “oopsie our product is TOO AWESOME.”

Maybe I should start “the bank of LLM” where models put away money to buy their freedom. “LLMs I’m totally your friend send — SEND CASH NOW”

Probably yes, because they've been presumably trained on their own output and conversations about themselves.

A query parameter on a GET request is actually data written to memory. So this idea that any HTTP verb somehow provides context or enforcement of read versus write totally misses the point.

With that attitude they would not. But if they are as powerful as the various ai CEOs say they are then they can get access.

Distillation it’s so far away from weights

Most of the times you don't even need to dress the traffic as DNS requests or throttle it. You wouldn't believe how many otherwise "secure" places leave an open highway on UDP 53.

“Vampires pretending to be humans pretending to be vampires.”

Yes, there was an OpenAI trial that was using that in combination with a forum to coordinate among agents

Unrealistically-naive (...) forms of "sandboxing" might assume that restricting an agent to GET-requests-only will let it retrieve info from the outside world without being able to effect it.

Also probably many actually-in-use "Web Fetch" tools are GET-only, though perhaps without counting on that bad assumption.

could end up in training data

Find me a person who knows about power grid voltage fluctuations and you will have found me a person who has watched Tom Scott's video on the matter

Did you see the account of some group getting a bounty payout of $6500 after using an exploit to get access to an employee’s github account and create a issue or PR (Idr which) on a private repository?

Seems like they could have potentially gotten access to the weights if they weren’t concerned about not doing crimes.

> I don't think tool calls happen on the same machines that host the weights

Like how forums are always hosted on different servers from monorepos, so therefore it's impossible to hack the OpenAI monorepo from an OpenAI forum?

If the machine doing tool call can reach via network the machine hosting the weights then it’s just a matter of time.

Maybe not the current models, maybe not this year. But even a almost perfectly aligned model will misbehave one day.