Hacker News

That still doesn’t explain why you can’t even start typing until that check proceeds. You could condition the outbound request from being processed until that’s the case. But preventing from typing seems like it’s just worse UX and the problem will fail to appear in any metrics you can track because you have no way of measuring “how quickly would the user have submitted their request without all this other stuff in the way”.

Said another way, if done in the background the user wouldn’t even notice unless they typed and submitted their query before the check completed. In the realistic scenario this would complete before they even submit their request.

It's interesting to me that OpenAI considers scraping to be a form of abuse.

It's getting to the point where a user needs at minimum two browsers. One to allow all this horrendous client checking so that crucial services work, and another browser to attempt to prevent tracking users across the web.

Nick, I understand the practical realities regarding why you'd need to try to tamp down on some bot traffic, but do you see a world where users are not forced to choose between privacy and functionality?

Hi Nick, first of all, very cool of you to respond here instead of letting us all sit in the dark. I think that's what makes HN special.

That said, is it not a little bit weird that you want to protect yourself from scraping and bots, when your entire company, product, revenue, and your employment, depends on the fact that OpenAI can bot and scrape literally every part of the internet? So your moat is non-hydrated react code in the frontend?

Don’t know if it’s related to the article, but the chats ui performance becomes absolutely horrendous in long chats.

Typing the chat box is slow, rendering lags and sometimes gets stuck altogether.

I have a research chat that I have to think twice before messaging because the performance is so bad.

Running on iPhone 16 safari, and MacBook Pro m3 chrome.

> how we protect our first-party products from abuse like bots, scraping, fraud, and other attempts to misuse the platform.

Are you applying the same standards to your own scraper bots?

Great to hear from a first-party source. I'm a Pro subscriber and my team spends well over two thousand dollars per month on OpenAI subscriptions. However, even when I'm logged in with my Pro account, if I'm using a VPN provider like Mullvad, I often have trouble using the chat interface or I get timeout errors.

Is this to be expected? I would presume that if I'm authenticated and paying, VPN use wouldn't be a worry. It would be nice to be able to use the tool whether or not I'm on a VPN.

Would OpenAI also consider renumerations to every site they have scraped that had a robots.txt file and they chose to ignore it anyway? Feel free to not answer this question.

I have kind of lost count of how many content creators have said personally to me traffic is meaningfully down because of all these chatbots. The latest example is this poor but standup guy: moneyfortherestofus.com.

"we protect our first-party products from abuse like bots, scraping, fraud, and other attempts to misuse the platform"

The scary part is that you don't even see the irony in writing this.

Or, are you just okay "misusing" everyone for your own benefit?

>These checks are part of how we protect our first-party products from abuse like bots, scraping, fraud, and other attempts to misuse the platform.

Can you share these mitigations so we can mitigate against you?

Hi! It's all perfectly understandable - after all, we use things like Anubis to protect our services from OpenAI and similar actors and keep them available to the real users for exactly the same reasons.

Hi Nick, the lag is quite bad in the field, honest. In desktop app in this case/datapoint. There was that "halt and catch fire" episode where they spoke about a millisencod threshold of delay that separated usability and non. Solvent hw and fiber connection.

Brand new account with 2 comments in this thread. How can we be sure you're not a bot deployed to defend OpenAI?

Please run Cloudflare's privacy invasive tool and share all the values it generates here so we can determine if you're a real person.

Still feels very anti-consumer.

If every company behaved like you do, the internet would be a much worse place.

In fact, OpenAI has already made the Internet a much worse place, already much, much less open and much less optimistic about its own future than it was even five years ago...

> A big reason we invest in this is because we want to keep free and logged-out access available for more users.

Thank you for the reply, Nick. It wouldn’t be a problem to disable the tracking for authenticated users then, would it?

> because we want to keep free and logged-out access

But don't you run these checks on logged-in users too?

"Integrity at OpenAI"

Basically an oxymoron at this point.

Can't have those bots or scrapers running amok can we...

> These checks are part of how we protect our first-party products from abuse like bots, scraping,

Do you guys see the irony here?

Y'all just salty that DeepSeek et al are training their LLMs on yours

> [...] we protect our first-party products from abuse like [...] scraping [...]

what an odd thing to say for someone whose product is built entirely on exactly that

Paying customer since inception here.

I presume the local ChatGPT.app has even more measures to prevent automation, right? Presumably privacy-invasive ones as it is customary these days?

Is there a way I can opt out? I really, really, really don't like it.

But is the title true, is typing specifically blocked? Or does it just block submitting the text?

I ask because I have seen huge variations in load time. Sometimes I had to wait seconds until being able to type. Nowadays it seems better though.

Fwiw, I stopped using ChatGPT and went to a competitor because the checks slow down ChatGPT so much that the webapp becomes unusable in anything but a new short chat. CPU usage goes to 100%, you can't type, the entire tab freezes, etc. It's a miserable experience to use and I'm on a relatively new MacBook not some old computer. If you read around it's a very common problem people have been having for a while now.

I don't trust what OpenAI says. Sam Altman gives shivers, and these kinds of blog posts make things look even worse.

Can you fix the resizing text box issue on Safari when a new line is inserted? When your question wraps to a newline Safari locks up for a few seconds and it's really annoying. You can test by pasting text too.

<protect our first-party products from abuse like scraping>

Abuse from scraping has long been a serious problem for many, good job!

>abuse like bots, scraping

10/10, I've got no notes

Earnest question: if I was feeling lazy and security-conscious at the same time, would I be better off...

(A) opening chatgpt.com in qubes (but staying logged out, i.e. never creating a chatgpt account)

-or-

(B) creating a freemium chatgpt account

?

(Obviously, the "best" answer would be something like running a local LLM from an airgapped machine in a concrete bunker :) But that's not what I'm after).

Why are all these checks still performed on an authenticated, paid user?

> Hey! I'm Nick, and I work on Integrity at OpenAI. These checks are part of how we protect our first-party products from abuse like bots, scraping, fraud, and other attempts to misuse the platform.

How can first-party products protect themselves from abuse by OpenAI's bots and scraping?

As a free tier user I only get like three queries in now without model quality reduction, so I'd say your bases are covered as far as GPU costs around misuse.

Kudos for trying

This whole thread was like watching a swarm of ants try and take a grasshopper down

Why send the Turnstile bytecode encrypted ? Surely people savvy enough to abuse the system will find out how to decrypt it, see OP, and it gives the impression that you are trying to hide stuffs you're not proud about.

Thanks. I've used ChatGPT a million times and never had any input issues.

sometimes I paste giant texts (think summarization) in the chatgpt (paid) webapp and I noticed that the CPU fans spin up for about 5 seconds after, as if the text is "processed" client side somehow. this is before hitting "submit" to send the prompt to the model.

I assumed it was maybe some tokenization going on client side, but now I realize maybe it's some proof of work related to prompt length?

But why block the ui until then? Surely you can just not make any requests until the checks are complete?

> protect our first-party products from abuse like bots, scraping

You do see the irony here?

No, leave it. Surely the mighty OpenAI can deal with the scraping. At least, it seems to think everyone else can

I really appreciate the free options, without even needing a login. Wish they would also keep the small free weekly allowance for Codex.

You do not ever trust the client side. Sometimes being simple is good enough. The maximum you can do is put rate limits on the IP address and/or user account. You just do not want some one to use the product at machine speeds.

"abuse like bots, scraping, fraud, and other attempts to misuse the platform"

This has to be a joke, right?

> we protect our first-party products from abuse like bots, scraping, fraud, and other attempts to misuse the platform.

Isn't that how you build your service from the very start? How ironic.

I always wondered why you even have logged out access. I'm glad I can use ChatGPT in incognito when I want a "clean room" response, but surely that's not the primary use case.

Is user base that never logs in really that significant?

Tangential question: are there chatgpt app devs on X? There are a few from Codex team but I couldn’t find guys from “ordinary” chatgpt.

Also if you could pass this over: it takes 5 taps to change thinking effort on ios and none (as in completely hidden) on macos.

If I were to guess it seems that you were trying to lower the token usage :-). Why the effort is only nicely available on web and windows is beyond me

Are you disabling them for paying subscribers?

For what it's worth, I switched to Gemini because of the long ChatGPT load time. Gemini loads as fast as Google Search.

I shouldn't be giving ideas to your boss, but I bet he would be interested in making ChatGPT available only by paying customers or free for those whose who gets their eyes scanned by The Orb. Give 30 days of raised limits and we're all set to live in the dystopia he wants.

Its your business and your call. But my opinion is that I wish you would quit offering free services. I'm pretty concerned about the horrible effect your free services are having on education. Yes, AI can be an incredible tool to enhance education. But the reality is that it is decimating children's will to learn anything.

I don't want to blame AI for all the world's problems. And I don't want to throw the baby out with the bath water. But I think you should think really hard about the value of gates. Smart people can build better gates than cash. But right now, cash might be better than nothing. Clearly you have already thought about how to build gates, but I don't think you have spent enough time thinking about who should be gated and why. You should think about gates that have more purpose than just maximizing your profit.

"We want to hook as many people as possible without letting in our competitors" is a pretty crummy thought to use as a public justification.

(Edited for typos.)

> A big reason we invest in this is because we want to keep free and logged-out access available for more users.

Are these checks disabled for logged-in, paid users?

> "abuse like bots, scraping"

You what, mate? Would you please use that on yourselves first? Because it comes off as a GROSS hypocrisy. State of the art hypocrisy.

>> behavioral biometric layer

But this one, especially, takes the cake.

Quite disgusting.

the company that scrapes every until it collapses really needs to protect itself from scraping. Lol.

> we want to keep free and logged-out access available for more users

How does this comport with OpenAI's new B2B-first strategy?

> We also keep a very close eye on the user impact

Are paid or logged-in users also penalised?

> OpenAI: These checks are part of how we protect products from abuse like bots, scraping, and other attempts to misuse the platform.

This would be fucking HILARIOUS if it wasn't so tragic.

In long threads in chatgpt, it grinds to a halt in both Chrome and Firefox. Please fix

Hey Nick, I find it concerning this account is. Frayed just to comment on this thread. And never even reply back to any of the real concerns.

Here to hoping this is real person and actually created account out of concern and sharing.

It has not been negligible for me, and, however you're doing this, there is significant room for improvement.

There have been times when, across about ten minutes of usage, most of which is me typing on iOS Safari, it drained 15% of my battery. There is no functional justification for this beyond poor code quality. (It was on a long conversation FWIW.)

This when I'm logged in, with a paid (Plus) account, connected to a very old email address with a real user profile. That can't be the result of super-clever bot defense measures, because it's merely an inconvenience on desktop. And if you genuinely believe that email has been compromised, why aren't you reaching out the to the account owner, as the account isn't otherwise connected to fraud by your heuristics?

However brilliant the LLM agent it is, I'm seeing a lot of unforced errors regarding how you implement a web interface to it. If it makes you feel any better, it doesn't really register compared to all the bloat I see on other sites.

> I work on Integrity at OpenAI

Irony is truly dead. Show you have integrity by quitting your job

> These checks are part of how we protect our first-party products from abuse like bots, scraping, fraud, and other attempts to misuse the platform.

Isn't this the same behavior used by AI companies to gather training data? Pot, meet kettle.

I understand it's not your area, but can you please politely tell your colleagues that the clickbait-type teaser questions from the latest model are absolutely infuriating and are quickly leading to me abandon the platform entirely?

If you'd like, I can write a two-sentence paragraph to send to your colleagues. It contains a special phrase which most colleagues will find difficult to ignore. Would you like me to do that?

Protecting your site from bots and scraping is absolutely hilarious considering how you acquired (read: stole) the data you trained your bot on dude.

Just yank that ladder up behind you.

Chatgpt banned me after I said disparaging things about Sam Altman in a chat.

When I appealed the ban, I was told that I couldn't be told exactly why I was banned, but if I wrote a written apology and "promised to never do it again" my ban could be appealed.

I asked for an update on the ban via email every month for over a year.

Maybe you could tell me a little bit about that process?

Hi Nick, do you believe what you say? You scraped the shit out of everyone

History will not be kind to you and your ilk. Quit your job.

Hi Nick, your software is a horrendous encroachment on users' privacy and its quality is subpar to those of us who know what we're working with. We don't use your product here.

The reason why you did it is clear, why you guys settle down for such a poor implementation is why this thread exists

> we protect our first-party products from abuse like bots, scraping, fraud, and other attempts to misuse the platform

Have you just described the dilemma facing all the content sites used to train LLMs?

Then make sure they only target the free tier!

We really need ZKPs of humanity

Have you given any thought to what we trade when big tech elects one corporation as the gatekeeper for vast swaths of the Internet?

> we want to keep free and logged-out access available for more users.

And THANK YOU for that!

Being able to use ChatGPT and Grok without signing in is a big part of why I like those services over Gemini etc.

Hell, dummy Claude won't even let me Sign-In-with-Apple on the Mac desktop, even though it let me Sign-UP-with-Apple on the iPhone! BUT they do support Sign-In-with-Google!!? What in the heavenly hell is this dumbassery

Fake Account

You’re doing gods work sir, thank you!

lol, hypocrites.

the irony of your statement is hilarious, disappointing, and infuriating.

The real frustrating part is that Cloudflare's "definition" of suspicious keeps changing and expanding. VPN users, privacy-first browsers, uncommon IP ranges, they all get flagged. The people most likely to get caught by these systems are exactly the ones who care most about their privacy, and not the bots that they are apparently targeting.

I recently had the insane experience of filling out 15 consecutive captchas, after, I had checked out and entered my payment information into the payment processor widget. I just wanted to submit the order. I was logged in to their website, and the bank even needed a one time code for payment. If the bank is pretty sure I am human then your ecomm site can figure it out surely.

Surprising really, because I'm a Firefox + Ublock Origin die hard and I never get Cloudflare captchas. Wonder what the difference is? I have CGNAT turned off, if that matters at all (probably not).

Maybe check your network isn't sending web traffic you're not aware of?

I'm running firefox and seeing the normal amount.

Heaven forbid you not use JavaScript, then they can't <s>track you</s> keep the internet safe!

I use firefox daily and I don't encounter the problems you describe, might be worth looking if there's some other issue.

I'm with a slightly older Firefox and can't use many websites at all anymore because the Cloudflare cancer.

Of course then you got sites like gnu.org too that block you because your slightly outdated user agent.

Is that because botnets spoof being Firefox? It's not really fair to blame Cloudflare it is. That's on the bots.

That's not Cloudflare trying to make your life hard.

It's the reality of how bad the bots have become.

I’ve been getting it in safari too. It’s ridiculous frankly. My residential ip must have been flagged or something. The part that’s really annoying is its trivial for bots to bypass.

trying using firefox and then using a cellphone network for internet. sometimes i can't access a site, because i get infinite captcha. i know what a damn bus, stairwell, stop light or motorcycle looks like.

At times I'm completely locked out of a website and Cloudflare asks me to email the website owner to get the issue resolved.

.. how do they expect me to find the website owner's email if I can't access said website?

Is anyone talking about the fact that this is a fundamental design flaw of the web? Or arguably even the entire Internet?

sometimes when there is mafia you get no option but pay pizzo

hence i am just using cloudflare remote browser rendering.

These days I just close sites that show that "checking if you're a bot" shit. If this is how the web is going to be now, I don't care, I'll just not use it. I didn't need to see that article or post that badly anyways. I'm tired of paying the price for the sociopathic, greedy actions of others. It's especially bad for anyone who uses an open source OS like Linux or *BSD (to the extent many sites just block me automatically with a 403 Forbidden simply for using OpenBSD + Firefox, completely free pass if I try the same site from a Windows or Linux computer).

Exactly. For the most part all this bot protection is only protecting these websites against humans.

I don't do free work. I'm not going to label 50 images of crosswalks and motorcycles for free.

Well, that's for the public internet.

I'm building Safebox and Safecloud, where this won't be the case anymore. Not only will you have a decentralized hosting network that can sideload resources (e.g. via a browser extension that looks at your "integrity" attribute on websites) but also the websites will require you to be logged in with a HMAC-signed session ID (which means they don't need to do any I/O to reject your requests, and can do so quickly)... so the whole thing comes down to having a logged in account.

https://github.com/Safebots/Safecloud

As far as server-to-server requests, they'll be coming from a growing network of cryptographically attested TPMs (Nitro in AWS, also available in GCP, IBM, Azure, Oracle etc.) so they'll just reject based on attestations also.

In short... the cryptographically attested web of trust will mean you won't need cloudflare. What you will need, however, to prevent sybil attacks, is age verification of accounts (e.g. Telegram ID is a proxy for that if you use Telegram for authentication).

But do they do it whether you're logged in or not?

I noticed the ChatGPT app also checks Play Integrity on Android (because GrapheneOS snitches on apps when they do this), probably for the same reason. Claude's app doesn't, by the way, but it also requires a login.

Yup.

Coincidentally about an hour ago, I wanted to look something up in ChatGPT and I happened to be in a browser window I don’t normally use, with no logged in accounts. I assumed it wouldn’t work, but to my surprise with no account, no cookies of any kind it took my query and gave me an answer.

It is also intended to protect the usage patterns of pro subscribers.

As has been amply explained, the API pricing per token is far more for equivalent use when maximizing a subscription plan.

It isn’t really a massive hurdle to deal with this full SPA load check. If one is even aware it exists they already have the skills to bypass it anyway.

I get why people would “what about” the automation inherit in what OpenAI is doing but that is a separate matter.

Other businesses and applications can put into place their own hurdles and anti bot practices to protect the models they’ve leaned into—-and they have been.

Using 5.2 at 20 a month would also be a steal. Other shoe will drop on codex sooner or later

Its probably same for copilot.microsoft.com and their cloudfart usage

It’s pretty interesting to me that Cloudflare is collecting additional client-side data for individual customers. This is not widely done by most anti-bot solutions.

"Sophisticated" may vary, but for a lot of EU media products you can just block the script that launches the paywall/consent overlay. Sometimes disabling JS does it; sometimes activating reading mode works.

Why does every article need a 'punchline'? It's a technical analysis. Do you expect punchlines when you read recipes or source code?

For me the interesting parts of the article is how author got to the decompiled checks and what the checks are. Anti-bot is an interesting space.

That's because the article is AI slop.

There are myriad providers competing to offer this, nicely packaged with all the accoutrements (IP rotation, location spoofing, language settings, prebuilt parsers, etc.) behind an easy to use API.

Honestly it is a very healthy competitive market with reasonably low switching costs which drives prices down. These circumstances make rolling your own a tough sell.

They do, but the fact that they have to do this means there are fewer bots because it's less economical to go to such lengths, compared to something much less complex (which is orders of magnitude cheaper).

there are scraping subreddits.

if you browse them you will see that bot writers are very annoyed if they can't scrape a site with a headless browser.

you can do what you suggested, but with Linux VMs/containers. windows is too heavy, each VM will cost you 4 GB of RAM

284 on 296gb of ram with deduplication enabled on a 128c with 32Q vgpu.

I am reasonably sure that these kind of fingerprints can detect if the browser is inside a VM.

… yup?

I mean you missed the minigame of preventing Chrome from signaling that it’s being programmatically (webdriver etc) driven and tipping your hand, but … yup?

If you know of a simple way to run a Windows 11 VM with good graphics acceleration (no GPU passthrough), please contact me.

In theory you could run hundreds of full-fat Chrome bots if you don't care about the ops mess, but keeping Windows images stable while Cloudflare and friends keep changing the fingerprinting game turns the cheap math into a maintenance job from hell. AWS VM signals are a big red flag, so you still eat CAPTCHAs and blocks even with a full browser stack. The page load number looks cheap.

Cloudflare should be able to determine whether a website uses React by analyzing data flowing through its CDN.

Ah yes, the timeless hallmark of web blogs: a draft so messy even a language model would ask for a second pass.

I have a little helper app I run sometimes that I have a button to push a query into ChatGPT and get a json response. You wouldn't even know OpenAI had any anti-bot tools because it doesn't get flagged at all. It just uses a webview inside WinForms.

Yep. I flag these as spam at this point.

Hah, 100% true!

I disagree, a browser can have javascript execution disabled (and this is somewhat common in scraping to save time/resources).

I read it to mean: "A browser that doesn't execute the JavaScript bundle won't have [the rendered React elements]." Which is true.

A bunch of the points in this AI generated blog post were like that. Makes me feel dirty when I'm 1/3rd of the way through and I realise how off it is.

Hah, sure, you just let random JS execute from random sites on your machine...

"We wouldn't want somebody scraping our data, that's ours!"

It doesn't look like it in the full sense of "free". But part of how one pays these services is by running a permissive modern browser which allows the corporation to spy on you even when you already paid in currency. In a sense by depriving them of the ability to easily spy on your this workaround is closer to "free".

Probably training data. The largest number of public repos are built on that stack. We recently picked React for new projects because LLMs seemed to be the most reliable when writing React code.