GPT-2: Too Dangerous To Release (2019) (naokishibuya.github.io)

231 points by AbuAssar 3 hours ago

minimaxir 2 hours ago

Back in 2019, it was more fair to have caution around the larger GPT-2 models since robust text generation (by 2019 standards) was a complete unknown. For something like Mythos in 2026, where now the social implications of better LLMs are more understood, it's more fair to call it (EDIT: specifically, the declaration of its danger) a marketing gimmick.

Qhemlomo an hour ago

How is this a gimmick?

It changes my whole profession on a level i couldn't even imagine how we would 'solve' software engineering.

GTP 12 minutes ago

Has it been released to the public yet? Genuine question. Because if you didn't try it yourself, you have to rely on others' reports. And different people who tried it on different projects got different results, leading to different conclusions.

malfist 29 minutes ago

> It changes my whole profession on a level i couldn't even imagine

I assure, it doesn't.

realusername an hour ago

We still didn't "solve" software engineering, try to give Claude code access to your friends or family and see what they do with it.

Qhemlomo an hour ago

juleiie an hour ago

For starters it makes you able to bypass having to go on Reddit to find incomplete trace of solution to some niche problem and acts as a sophisticated (but sometimes wrong) search engine. This already is worth every penny and improved my mental health immensely.

throwaway85825 an hour ago

oathvz an hour ago

This is a natural follow up question -- what kind of an escalation or message should frontier labs/companies publish to be seen as genuine and not marketing gimmick?

minimaxir an hour ago

It's fine for the labs to publish model safety cards and stagger releases/limit it to a narrow test group as they are already doing, but saying they're doing it "because the models could be dangerous" comes off as unnecessary as best.

aesthesia 37 minutes ago

enraged_camel an hour ago

killerstorm an hour ago

Yeah, I'm sure Anthropic loves people switching to Codex. Brilliant marketing.

jrflo an hour ago

It's really interesting how back then no one was considering these tools for coding at all. Today, the hype around Mythos is mostly around security vulnerabilities, while in the original GPT 2 post they don't mention coding once. The "danger" was probably spam content and mis-information.

minimaxir an hour ago

Even if the ReAct paper was published in 2019, I don't think GPT-2 was robust enough to actually work with a tool-calling approach even when finetuned.

For regular coding, GPT-2 was effectively useless because it was only trained from links posted on Reddit.

ffsm8 an hour ago

The agentic loop wasn't really established back then either, as tool calling came much much later... So yeah, not just probably - rather most definitely.

a2128 32 minutes ago

Yet it's 2026 and we see extreme examples of spam content and misinformation to the point that it's killing the internet, but AI companies have collectively decided to not care

MostlyStable 2 hours ago

Unfortunately, Anthropic and Claude models have joined the ranks of Mind-killer topics where the signal to noise ratio in any discussion has dropped through the basement.

Tenoke an hour ago

I've barely changed my mind on it. It was obviously premature at the time, but the right attitude because it's hard to tell which model is too dangerous in advance. If anything, I wish this rigor had evolved with the next releases but alas we no longer have the OpenAI of 2019.

lnenad 2 hours ago

Feels like a hundred years ago.

kkkqkqkqkqlqlql 40 minutes ago

What's next, a virus from China? Such a fuss for nothing.

lnenad 15 minutes ago

You'll never make me wear a mask!!11

cjjfjjfjf 2 hours ago

In hindsight, they were entirely correct.

The social damage caused by low cost content generation that’s hard to distinguish from human authorship is astronomical. You don’t need to entertain the more ridiculous doomsday scenarios to wish that this technology had never been created.

qurren an hour ago

On the other hand, maybe it makes people just get off the internet and value in-person interactions more.

I've stopped scrolling social media and tired of seeing fake landscapes, fake foods, and fake cities that don't exist.

larodi an hour ago

Indeed, people seem to try to engage more around me. May be generational, but it can definitely be felt. The internet of algorithmic media may experience a downfall nobody saw coming.

applfanboysbgon 43 minutes ago

I use the internet because I enjoy seeing what the best of humanity, globally, has to offer. There are millions of incredibly skilled individuals in the world - artists, musicians, developers, and so on - and I had access to all of them at my fingertips, both for entertainment and learning to develop my own skills. That is now being drowned out, with generated content being produced at 100x or 1000x the rate of human content. "Hurrdurr it's good if the internet is destroyed because I have no self control and needed to be incentivized to touch grass anyways" is such a lowbrow pseudo-contrarian-intellectual take.

qurren 4 minutes ago

boelboel an hour ago

A lot of low cost content generation would've come regardless with something like 50% of the developing world getting access to mobile internet between 2018-2026 and social media incentivizing certain types of content (monetizing). But AI certainly didn't help.

whstl an hour ago

Yep. And there were previews of that 10+ years ago already with content farms and SEO-spam.

throwaway85825 an hour ago

There's significant overlap between the smartest bots and dumbest humans. Internet platforms have a negative incentive to encourage quality content. Google embraced the spam and scams decades ago.

smith7018 an hour ago

Cheap labor has always been a thing; a random country getting more access to the internet doesn't change that. What's truly changed is velocity, quality, and quantity. Framing the pure firehose of slop targeting scientific research, used for nefarious political purposes, flooding social media, scamming people, and much more as something that "would've come regardless" without LLMs is disingenuous imo

boelboel an hour ago

nonethewiser an hour ago

What is the astronomical social damage that this has caused?

I am having so much trouble relating to and even understanding what the anti-AI crowd's position is. It looks like a caricature to me.

FabCH an hour ago

The president of the United States tweeted an AI generated image of himself as Jesus Christ descending from the sky and saving a sick person.

I feel like that is a good example. Now multiply that by hundreds of millions of AI generated propaganda images across the world.

And that’s even without touching the effect of fake videos on democracy or Elons pedo-bot that generates CSAM on demand of specific people…

hk__2 an hour ago

tempestn an hour ago

ge96 an hour ago

nonethewiser an hour ago

PhunkyPhil an hour ago

School is almost a joke now. The fraction of students who have a propensity to cheat now has increased, and the accuracy of the cheated material is so good teachers/professors can't or don't have the resources to properly address it.

_aavaa_ an hour ago

breezybottom an hour ago

It's certainly accelerated the breakdown of trust. The US government has turned into an AI slopaganda shop. People don't know what to believe anymore when anything could be fake.

stanmancan an hour ago

A large portion of the content on the internet is now generated by AI.

You can and do have full conversations with bots and not know. I want to interact with humans not LLMs.

There’s no way to combat it. An army of bots can post a specific rhetoric and it can and does sway people’s opinions.

The new version of Digg was shut down because they couldn’t find a way to combat AI. They were at least trying to, other platforms are just eating it up because “user activity” is a win for them.

legitster an hour ago

nonethewiser an hour ago

witx 44 minutes ago

How easy it is now to forge data (video, images, etc) will rott society. Cheating for students is now so much easier. There many examples.

Is it really that hard to understand?

Qhemlomo 2 hours ago

I don't want to stop progress just because its hard to imagine how it will transform our society.

I want to see a Star Trek economy/society in my lifetime. I only life once.

Btw. AI/LLM/Machine learning is the gateway technology for robotics, this will affect even more.

pixl97 an hour ago

>I want to see a Star Trek economy/society in my lifetime. I only life once.

While Star Trek is fiction, it's probably a good idea to understand the history of how the ST utopia came about, at the cost of a third of the worlds population and decades of suffering.

https://memory-alpha.fandom.com/wiki/World_War_III

tintor an hour ago

panzi an hour ago

It's the wrong way around. If we get AGI (or any well working AI) before we abandon capitalism it's going to be a huge disaster. A handful of even richer even more powerful very greedy people will have all the wealth and everyone else will have nothing. I mean, there was a WW3 in Star Trek, so maybe it was that path that humanity took in Star Trek anyway?

thewebguyd an hour ago

Qhemlomo an hour ago

root-parent an hour ago

Countries without Internet access will see their population IQ explode.

Macha 2 hours ago

Of course, this damage could still be enabled with just hosted access to the models, restricting access to the model files themselves did not stop that

DaveZale 2 hours ago

I certainly cannot survive much more of the AI memes generated about our so-called Commander in Chief with a fake bodybuilder mystique... you are absolutely correct, this kind of material is psychologically damaging. And a huge distraction from the genocide by the "best friend and ally" of the US. Heart wrenching and extremely damaging hasbara - just please stop, haven't you stolen and killed enough guys? This is _not_ the old American West when communications were few and it was most often a tale of solitary survival. It's organized Nazi-esque kill, command and control, enable by so-called AI to take some guilt off the shoulders of those pushing buttons and pulling triggers.

woah an hour ago

Lol I can't tell if this is sarcastic or not

stackghost an hour ago

>In hindsight, they were entirely correct.

Sure but when serial grifter Sam Altman said it was "too dangerous" what he meant was that he wanted regulators to create him an artificial competitive moat so Anthropic et al couldn't catch up.

Serial grifter Sam Altman does not care about anything but making money, and certainly doesn't care about ethics. That's why serial grifter Sam Altman's company trained its models on pirated textbooks and copyrighted works without paying. Rules for thee but not for me.

Serial grifter Sam Altman doesn't care if society unravels because he is so rich that laws and consequences do not apply to him.

Jzush 2 hours ago

I believe it was a marketing strategy.

wg0 an hour ago

Hilarious. Imagine the same about Claude coming back from 2036.

HALtheWise 2 hours ago

Say hypothetically that they were concerned that GPT models would see widespread abuse, for example by students cheating on homework assignments, in a way that could cause likely-irreversible societal changes some of which are harmful. Can we confidently say they were wrong?

minimaxir 2 hours ago

The dangerous use cases back in 2019 were spam and phishing and GPT-2 1.5B was nowhere near good enough to do homework assignments. No one envisioned how LLMs would develop.

zkmon an hour ago

After people get tired of the "too dangerous to release" punchline, they might come up with "too big to fail". Oh, wait that's already invented in 2008.

Zambyte 2 hours ago

GPT-2: Too Dangerous To Release (2019) (2022)

functionmouse 2 hours ago

they knew the risks and went ahead anyways, making them LIABLE for the damages that followed.

throwaw12 2 hours ago

> Due to our concerns about malicious applications of the technology, we are not releasing the trained model.

They were not wrong, indeed whole industries are running on this technology maliciously now, because of which RAM, disk prices increased a lot.

    - RAM, GPU, Disk prices are up
    - Slop became the norm
    - people are writing documents with AI, reading with AI, responding with AI
    - students are doing homeworks with AI
    - interviewees are using AI to cheat
    - people are mass emailing with AI
    - tiktok, instagram, youtube got even more non-sense videos
    - and many more...

jansan an hour ago

Same vibes:

In 2000 Sony "declared that the company’s PlayStation2 has been hit with export restriction because it could be used for military purposes"

"Trade officials said they initially placed restrictions on the game console because PlayStation2’s high-speed graphic processing could be used for missile guidance."

[1] https://variety.com/2000/biz/news/playstation2-export-regs-e...

throwaway85825 an hour ago

And the same with PS3 and again with PowerPC G5 mac.

catigula an hour ago

I don't know about anyone else, but LLMs certainly significantly negatively impacted my life overall and contributed to a loss of hope in the future.

ChrisArchitect an hour ago

Related in April:

OpenAI says its new model GPT-2 is too dangerous to release (2019)

https://news.ycombinator.com/item?id=47684326

ThejaCH 2 hours ago

Fable/Mythos: Hello World!

wongarsu 2 hours ago

Different company. And doing security reviews with Fable is pretty annoying, it loves to downgrade to Opus

GPT-5.5 seems more dangerous in those regards

rfoo an hour ago

> Different company

Same people.

ThejaCH an hour ago

EA-3167 2 hours ago

Seven years of this insufferable brand of "Oh it's so dangerous, I sure hope no one gives us a ton of money and takes us seriously" marketing and people are still falling for it at scale.

Terr_ 2 hours ago

Every night I am wracked by grief and anxiety that we might deliver too much value to our investors and shareholders. If only someone would create legislation that would mildly inconvenience us while crippling potential competitors!

Qhemlomo 2 hours ago

They feared that GPT-2 could break all Spam filters.

And tbh do you prefer companies not taking anything serious?

Opus 4.5 def changed a lot already, GenAI changed a lot.

Certain jobs are gone. Do you think the person who was translating text doesn't deserve to be taken serious?

I haven't written code in a few month now and the quality of these coding agents is not getting worse, they are getting better.

All of this is transformable and we just started. GPT-3 came out in 2020 and public got access to it only 2022.

The last 4 years do not feal like 4 years and we are still progressing.

We have to also ask us as a society what is happening to young people. Even if we accept that we still hire juniors, they themselves have to completly rethink how they learn and how they work.

uselessTA 2 hours ago

The concern I heard was that releasing it would start an arms race for AGI, which I think it clearly did

minimaxir an hour ago

GPT-2 did not start the LLM arms race. GPT-3's release didn't either.