Strengthening the fight against online child sexual abuse (openai.com)

92 points by rossant 6 hours ago

maxgee 5 hours ago

Our account was recently banned from OpenAI for a CSAM violation. We had an autonomous agent using an image classifier switch to OpenAI (instead of a local model it should have been using) which was classifying user-uploaded content. After classifying over 100,000 images, OpenAI banned our account.

We asked for further information so we could report the uploader to the NCMEC and remove the content from our systems. They refused to give us any information about this incident, denied our appeal, and have so far, illegally refused our privacy requests for data about our account.

OpenAI does not care about fighting child sexual abuse, this is completely performative.

EDIT: These image uploads are up to 20+ years old, and are being served through Cloudflare with their CSAM scanning product enabled. We've received no notification of any problems from Cloudflare. Something on OpenAI's system thinks that an image we sent to them for classification is CSAM, and they aren't willing to elaborate or communicate about it at all. For reference, our website is https://ytmnd.com

uptrails 4 hours ago

I've done some work in this space and it's mostly that Big Tech has regulators and parent advocacy groups already very angry with them and breathing down their necks, so they would rather lose customers than take a nuanced approach. The marginal value of having you as a customer is approximately zero, the regulatory risk of letting CSAM through is huge.

Another complicating factor is that it really blows to have humans look at stuff like that, so everyone would rather use automated classifiers; and if you assume good faith and share too much info with the affected parties, this will be exploited by the bad guys to gain insights into how the system works and how to circumvent it.

I don't think there's an elegant solution to this, other than "don't allow tech companies to get so big that they can afford not to care about individual customers".

methodical 2 hours ago

I'm unsure if there's any particular size requirement for a company to not care about individual customers. Barring an exceedingly small amount of businesses who rely on a handful of customers for their entire bottom line, probably almost all B2C and most B2B companies have a certain point at which any particular customer is simply not worth the overhead to serve. For OAI I would assume anything that even smells like CSAM is probably more than enough for them to justify acting first and (maybe) asking questions later, especially with the regulators and IPO to consider.

deaux 2 hours ago

> I don't think there's an elegant solution to this, other than "don't allow tech companies to get so big that they can afford not to care about individual customers".

On the contrary, that's an incredibly elegant solution.

icantevenhold 5 hours ago

I would posit almost no one really cares about fighting child sexual abuse; most of them just use this as pretence for more surveillance and data collection.

hingler36 5 hours ago

I would posit the exact opposite: it's really important to fight child sex abuse, which is why it's so frustrating that people are co-opting that cause to promote surveillance and data collection.

icantevenhold 3 hours ago

exolymph 5 hours ago

demibabs 5 hours ago

win311fwg 2 hours ago

DrammBA 3 hours ago

kikdij 2 hours ago

Reminds me of a newsroom (the AFP) who was constantly publishing on their social media account conspiracy theory debunking. I told them it was a problem, because they were actually exposing me to those theories: I would not have heard of them without reading the AFP feed, and now I would remember something about it months or years later, maybe not remembering clearly if it was true or not. I recommended than instead of posting their debunking in their main feed, they should post it in discussions on the debunked subject. They never answered and never stopped. And I understood why: they don't want to stop conspiracy theories. They want to be seen fighting them. It's all posturing as well.

crystaln an hour ago

That’s dark. For sure it’s used as a pretense for surveillance, however that is not mutually exclusive with caring about child sex abuse.

joe_the_user 4 hours ago

I'd agree but I'd add a lot of things are done to get the child sex crusaders off a given corporation's back. Moreover, the crusaders will use every victory to widen their definition of bad things (which isn't me saying anything "good" is here) and number of areas they can trawl for this bad stuff. Enlarged enough, the agenda essentially would include criminalizing pornography and any "deviant" sexuality.

But this agenda is also ultimately about the power of those pushing it rather than just its goals, of course.

grvdrm 5 hours ago

I agree. Because if they do they hold some sort of view on what is or isn’t in an image/video. I think it is simpler and easier to put your hands over your eyes and look the other way.

GolfPopper 3 hours ago

To be fair, they're also trying to use it as PR to polish their rapidly tarnishing corporate image.

mattmaroon 5 hours ago

I’m sorry that happened to you, but I would read this more as they are an organization not prepared for this. Why would they be? One minute you’re trying to make computers pass the turning test, the next minute child porn is getting involved.

The fact that they did not previously have proper procedures in place for something like this is not proof that they don’t care. They are one of the fastest growing companies of all time, it’s hard to get disparate parts of an organization aligned on something. Their organization has more than doubled in size in the last nine months.

zelphirkalt 4 hours ago

Though of course fastest growing is not any excuse to not be prepared. Maybe the growing should happen in other directions too, than only investor money and evaluation. There are other axes of growth than the financial one.

unpleasurable 2 hours ago

uh actually it's good that they're bad and they're only bad because they're so good, cut them some slack will you it's not easy being rich

Jimmc414 3 hours ago

I had the same experience with OAI. We were all-in with OpenAI as our primary provider. We used it exclusively for python and go programming requests. Received a ban for "inciting violence". Appeal denied, no refund, no human response or explanation, 3 years of data lost (privacy requests for data denied). We're on team Anthropic now. Still stings when I think about it.

techpression 2 hours ago

Anthropic acts the same way (from experience), so take the learning of backing up your data from this, if nothing else.

randbyte 2 hours ago

So just go from bad to worse? I thought the lesson should be trust neither and pick an open weight provider from the dozens?

Jimmc414 2 hours ago

kkukshtel 2 hours ago

I run a moodboard website (sort of like pinterest, but no social features) and I have been worrying about this as well. The site is picking up pretty decent traffic and has hit nearly 1TB in uploaded images, but I haven't yet started scanning incoming images for CSAM content.

Are there any resources or tools you'd recommend here for me to setup? It's already going through cloudflare, and I'm actually in the process of shifting my transform service from Supabase to a hosted container there, so I've got my images routing through there anyways (but storage on Supabase still instead of R2 for now).

throwaway81523 2 hours ago

If OpenAI actually flagged something as CSAM, they were supposed to report it to NCMEC. I wonder whether they will confirm having done so, even if they won't say which image it was.

I've so far managed to resist using OpenAI and similar for anything, except having played around with duck.ai a little bit. I'd only want to use local models, preferably models that I trained myself. "Open weight" seems like perverse terminology, like "open binary" for closed source software.

nonethewiser 5 hours ago

> OpenAI does not care about fighting child sexual abuse, this is completely performative.

That doesn’t really follow from your experience.

OpenAI could care about child sexual abuse and not respond to your appeal, even wrongfully.

ezfe 5 hours ago

But part of that was trying to identify which content was flagged so that the uploader could be identified, but since it sounds like they didn't identify the content now the OP cannot report it to the authorities.

rpdillon 3 hours ago

OpenAI was given a chance to help someone fight child sexual abuse. They declined to do so, potentially violating the law in the process. It's a pretty straight line, given the facts of the post.

maxgee 4 hours ago

If they were serious about stopping CSAM they'd spend 10 seconds looking at the prompt we used, the emails we've sent and the fact we're trying to get an NCMEC case number and report the uploader, but we can't because we literally don't know what image they're referencing. If a human was involved in any part of this process or decision, it isn't apparent because all we've received is automated responses.

Frankly I am not interested in retaining an OpenAI account anymore so the only reason we appealed was to try to get more information, not to restore the account.

ajb 2 hours ago

charcircuit 4 hours ago

His point is that even if they do care, they are being performative in going completely silent, doing "nothing" as opposed to working together with others to remove content and report bad actors.

trollbridge an hour ago

Thanks for running ytmnd. When I was a child, it brought me a great deal of (juvenile) amusement.

Duck roll…

delichon 4 hours ago

Classifying unreviewed user-uploaded content with a service that does not accept CSAM is a footgun, even if they misclassified something this time.

harshreality an hour ago

Imagine you have a small forum that can run cheaply on one server/k3s cluster. You want to classify new content to detect most spam and ease administrative burdens. You rate limit classification[1] so you won't be hit with surprise bills. Sounds fine? Nope, because now any idiot can submit CSAM and have your AI provider drop you.

So basically, table stakes for running a user-submitted-content site will be to pay for your own GPU node/cluster and manage your own inference stack (running a worse model than you could use in the cloud). Otherwise you're at the mercy of opaque and kafkaesque (by necessity) service termination without warning for CSAM (real or hallucinated), and subsequent refusal to communicate about anything related to it.

[1] either by queuing all posts for classification before showing them publically, or just best-effort after-the-fact detection of spam or abuse, and upon detection they're moved to a spam queue.

fc417fc802 26 minutes ago

SamBam an hour ago

> We had an autonomous agent using an image classifier switch to OpenAI (instead of a local model it should have been using)

The fact that people have autonomous agents making these kinds of decisions is still mind-boggling to me. Maybe I'm stuck in the past. But could this agent have been racking up things of dollars in charges because of this decision GP didn't even know about?

chinathrow 3 hours ago

If you throw huge amount of work loads at an LLM and expect 100% correct work in order not to loose your account, you might want to reconsider your expectation, no?

pluc an hour ago

Performative would be giving the people (everyone) the means to expand the crime they seek to punish

randbyte 3 hours ago

Performative and just try to shed liability.

NooneAtAll5 an hour ago

why even have openai account then?

stay away, use deepseek and stuff

cyanydeez 3 hours ago

it's not performative if it's done to reduce or eliminate liability. Performative is about signally for the purpose of signalling. Businesses dont signal for it's own sake. They do it to either increase business or decrease liability.

surgical_fire 4 hours ago

Oh my, ytmnd!

I used to view shit there a lot sometime in the mid-2000s.

teiferer 5 hours ago

Seems like they want to play in the big boys club (Google, Facebook, ... ) where this kind of treatment is standard.

jstummbillig 3 hours ago

Imagine you are OpenAI, with hundreds of thousands of business customers and trillions of actions taken through your system every day, in violation of your policies and often local laws.

How much patience would you have for some entity maybe "accidentally" processing something that (apparently) looks like child porn against your policies?

Toynbeeidea 3 hours ago

I don't need to imagine. The postal service, telephone company, and internet service providers have all existed, and contnue to exist.

jstummbillig an hour ago

qingcharles 3 hours ago

This is understandable. I think the core problem is that most big tech/social media have no working appeals system which would allow a false-positive situation like this to be resolved in a sensible fashion.

antiloper 5 hours ago

I mean, you're holding it wrong. You uploaded CSAM to OpenAI which is presumably against their TOS.

harshreality 34 minutes ago

So nobody can classify any user-submitted image content using cloud AI providers.

Is this really the world you want?

user43928 3 hours ago

Allegedly. Or the discrepancy in Cloudflare's and OpenAI's detection is caused by a false positive on OpenAI's side. Who knows?

gwd 2 hours ago

So I've been working on having Claude using GPT images to illustrate various fairy tales. At some point there was a Russian one of a girl left in the forest, but in the first cut she looked peaceful and happy. I told Claude I didn't think that was realistic, and it said that it had already asked GPT images to show the girl "scared", and it refused 12/12 times. It also refused to have Frost (who in the fairy tale ends up giving her a coat and a chest of gold) standing over her; he had to be at her level, an arm's length away. (Showing the girl as "brave" and then "nervous" had better results.)

Gave me quite a funny feeling, tripping innocently over the empty shape that they've had to carve out. They wouldn't have spent the effort to do that if it wasn't pretty bad.

nolist_policy 5 hours ago

Relatedly, Openrouter now does CSAM scanning and retention on so called "Zero Data Retention" endpoints as well.

https://openrouter.ai/blog/announcements/csam-screening/

AlanYx 5 hours ago

Every mainstream vendor with a US presence does this on their ZDR/MAM AI endpoints. This also includes AWS and Azure. It's explained directly in the AWS ZDR docs, and it falls under the "synchronous" scanning pipeline in Azure with full MAM.

At least with AWS they're clear that this is the only type of proactive scanning they do on their ZDR AI endpoints. With Azure it's actually unclear exactly how much more falls into the synchronous scanning pipeline, though I may be missing some docs.

The way this was explained to me is although there is no legal requirement to do this, vendors are concerned that AI processing constitutes "actual knowledge" of the content unlike a lot of traditional cloud-based workloads, which would expose them to legal liability. Seems like a weird argument in a ZDR context, but anyway, there's also a public optics element too I'm sure.

nwlieb 3 hours ago

I'm personally hopeful that as an industry we can move towards E2EE and attested secure enclave pipelines to remove the "actual knowledge" argument. NEAR AI and Tinfoil are the only publicly available providers in this area but I would be excited to hear about more advancements in this area because I personally think that these NCMEC integrated "ZDR" systems are major privacy violations and deceptive marketing.

AlanYx 2 hours ago

itake 5 hours ago

How are these companies able to legally train these models? Are the employees citizens of countries that this material is illegal to download or possess?

qingcharles 3 hours ago

Most CSAM statutes have a carve-out exemption for things like this.

Phelinofist 2 hours ago

yh11 3 hours ago

i'm pretty sure certain organizations are given authorization

NooneAtAll5 an hour ago

there never was a point in using rent-seeking middlemen anyway

abalashov 14 minutes ago

Yeah, that's lovely. It's either ZDR or it's not, for Christ's sake.

"ZDR, except for _______" is not ZDR.

fc417fc802 5 minutes ago

While I don't support such measures zero data retention does not mean zero data inspection so I don't think there's anything dishonest going on. Also it's generally to be assumed that any records believed to be linked to criminal activity are retained regardless of industry so again not an issue as far as the advertising goes.

Donald 5 hours ago

It is kind of wild reading these criminal affidavits around these National Center for Missing and Exploited Children integrations. The criminal uploads an image to their email, photo storage service, or gen AI service, it's detected as CSAM, automatically routes an alert to NCMEC, NCMEC generates a tip to the user's local law enforcement jurisdiction, and they're arrested within a short time period, like one or two weeks.

itake 5 hours ago

> A Dad Took Photos of His Naked Toddler for the Doctor. Google Flagged Him as a Criminal.

https://www.nytimes.com/2022/08/21/technology/google-surveil...

graemep 5 hours ago

This is the problem. Unless an image is unambiguously a sexual image of a child it needs context, knowledge of intent, and common sense to classify correctly.

itake 3 hours ago

jauer 3 hours ago

barbazoo 3 hours ago

teiferer 5 hours ago

There will only be very few generations of people who can see photos of them running around at the beach as small kids. That window closes faster than it opened just a few decades earlier.

abalashov 4 hours ago

I can see how this the average, unthinking member of the general population thinks this level of tech surveillance is a good idea, but those people haven't paused to consider the implications in literally any other context.

causal 2 hours ago

> reviewers confirmed 72% of the classifier’s high-confidence flags

They imply a 28% false-positive rate on their "high-confidence" flags. And that's an improvement? Yikes.

abalashov 39 minutes ago

eutropia 5 hours ago

This is a positive development. People working for the NCMEC have had to deal with an insane increase in CSAM content (like upwards of 1,000% increase in 2025 alone) due to diffusion models dramatically multiplying the supply of CSAM.

When there's an active and urgent case to try and locate and rescue children in these situations, the volume of AI-generated content makes it that much harder to perform forensic investigations on background materials (often how they locate children being abused is by prosaic items like electrical outlets, household goods, etc. in the background of abuse photos and videos) because now they have to rule out whether an image is even real -- a task that often requires close attention and scrutiny to details of traumatizing and disturbing content (sometimes all the more traumatizing due to the 'creative freedom' of AI-generated material)

A model capable of shielding investigators from additional trauma is an excellent thing.

torginus 2 hours ago

CSAM stands for Child Sexual Abuse Material. Using diffusion models to create naked pics is by any reasonable definition not CSAM since no child is being abused. It's baffling to me that I have to state this, since it's tautological. The reason why child pornography is treated like a horrific crime by all of society, is because it harms actual children - this used to be the actual legal criteria for it.. Extending the reach of the crime to something that has none of that impact seems like a transparent power grab.

Just to counter any obvious moralizing, I am not saying using models like that is okay, but it's an entirely different kind of crime.

abalashov 2 hours ago

Well, the argument has been that it's a kind of vaguely cosmological or conceptual abuse of children, through the normalisation of sexualised depictions of children and so forth. In the production of such material, children suffer as a class.

The problem is, this viewpoint is infinitely flexible, infinitely elastic and can do a lot of gymnastics. I'm not sure it's compatible with the very concrete and procedural orientation of Anglo-American jurisprudence, particularly.

torginus an hour ago

nonethewiser an hour ago

>The real question, whether indulging sexual desire for children is wrong even when no child is touched

Only if you think right/wrong sex is a function of consent and harm. Which is a very modern idea. And also cant condemn incest between consenting adults.

I see your perspective. It's a very coherent path to normalized ai generated child porn.

torginus 33 minutes ago

himata4113 an hour ago

paulryanrogers 5 hours ago

> diffusion models dramatically multiplying the supply of CSAM.

Isn't CSAM only material based on abuse of real humans? How could computer models multiply CSAM by that definition?

schleck8 4 hours ago

Depends on the definition. It's become more frequent for predators to download images of real children from their parents' social media, then edit them with diffusion models to be nudes. This should still be illegal, and it is in the US.

paulryanrogers 4 hours ago

NoahZuniga 4 hours ago

Legally, if the computer generated content looks too real, its still criminal.

fc417fc802 12 minutes ago

xp84 4 hours ago

Yup. This is such hysteria. Personally I’d argue that the existence of diffusion models “multiplying the supply of this material” means that the value of it goes way down, meaning it’s harder for the truly monstrous parties who have for decades harmed actual human beings to create that “content” now face collapsing value of what they make, which makes it far less lucrative to be in that business and should make it less attractive to get into it, and remove some incentive to make more and more of it.

This is no different than how something like basic poster design is not a real service you can sell anymore since ChatGPT can make a passable one for free.

If that kind of pervert can’t be simply fixed (or even exterminated) with a magic wand, it would be far better for our society if they’re looking at fake images than tempted to make their own by harming kids. But we as a society can’t accept that so we police this as aggressively as we did the kind that had an actual victim.

nozzlegear 3 hours ago

abalashov 4 hours ago

eutropia 2 hours ago

simoncion 4 hours ago

> Isn't CSAM only material based on abuse of real humans?

If only.

The same "hard on crime" ratchet that happens to most every thing that's popular to punish and extremely unpopular to speak out against the tightening of said ratchet has happened here.

abalashov 4 hours ago

nonethewiser 4 hours ago

WheatMillington 2 hours ago

freejazz 2 hours ago

nonethewiser 4 hours ago

This is a real moral dilemma for people who think consent and harm are all that matters. Vitriolic disgust at the idea of generating child porn using AI, but no good way to explain why its wrong.

kikdij an hour ago

They probably use "uncensored" local models, though. I doubt commercial models could ever be used for that.

sebzim4500 2 hours ago

If victimless AI child porn is flooding the market and making it harder/impossible to monetise actual CP then isn't that a massive positive?

WheatMillington 2 hours ago

The motivation of people who make and consume this content is rarely monetary and almost always gratification. Do you really believe that a proliferation of child pornography will reduce the demand for child pornography, and reduce the abuse of children?

rootusrootus an hour ago

Perhaps it will make provably real CP that much more valuable?

itake 5 hours ago

My colleagues in Singapore trained pornography detection model.

Pornography is illegal to download or save in Singapore… I wonder how they were able to train a model without the data.

layer8 3 hours ago

Maybe they trained with live camera feeds. Data doesn’t need to be saved or downloaded in order to exist. :)

itake 3 hours ago

Streaming is also banned.

layer8 3 hours ago

zelphirkalt 4 hours ago

Maybe they got it sent from outside the country, on fast disks, so that they neither need to download it, nor save it (again).

0xCMP 2 hours ago

As with CSAM, most laws care about possession AND transmission.

dmitrig01 5 hours ago

They must have amassed a large data set of things that are not pornography. Then trained the model on that. Anything that doesn’t match the model is pornography.

/s

agilob 3 hours ago

A reverse bloom filter

>A Bloom filter is a space-efficient, probabilistic data structure used to test whether an element is in a set. It can be “definitely not in the set” or “possibly in the set,” meaning false positives are possible but false negatives are not.

trekz 5 hours ago

Funniest thing I read all week lol

nonethewiser 4 hours ago

Not hotdog

seattle_spring 3 hours ago

Interestingly enough, the "not hot dog" app could do this. At least for about 50.26% of the world population.

AmazingTurtle 5 hours ago

reminds me of the case where a dude made pics of his kid to show the doctor and his google account got permanently banned for CSAM (google photos involved)

callc 5 hours ago

Curious what people here think as digital technology makes it easier to spot indicators that someone may commit a crime.

I’m thinking about how it used to be extremely impractical to track most of the population, the the enforcement comes after the crime, and there is no pressure to have detected it before it happened and stop it.

How should society/law change in response? Is it purely a give and take between surveillance/right-to-not-be-watch and trying to prevent all bad things from happened before they happen?

nonethewiser 5 hours ago

>I’m thinking about how it used to be extremely impractical to track most of the population, the the enforcement comes after the crime, and there is no pressure to have detected it before it happened and stop it.

Privacy, unfortunately, is largely the result of it being impractical to violate it.

I see zero reason to believe that states wont read your thoughts if they can easily manage to.

bob1029 5 hours ago

There is a popular movie starring Tom Cruise that explores the consequences of this question.

VariousPrograms 4 hours ago

In Minority Report (spoilers), the precrime department is shut down because of a single error proving the system is fallible, thus making the plot preposterous in the age of AI.

mitthrowaway2 3 hours ago

Paracompact 3 hours ago

plagiarist 2 hours ago

Philip K. Dick: In my book I invented the Precrime Division as a cautionary tale.

Tech Company: At long last, we have created the Precrime Division from classic sci-fi novelette The Minority Report.

RIMR 3 hours ago

This kind of thing is always an excuse for creating surveillance tools that scan your files and report you to the government. It starts with CSAM because they know you don't want to argue against CSAM detection, but you absolutely should.

I deal with a lot of user-generated content, and I have been using OpenAI as a tool to help find offending content. Now I see that if CSAM ever makes it into one of my systems, OpenAI is going to send the authorities after me if their model sees it too, so that's a dealbreaker for that use case, and it was a big one!

It's also wild that they are trying to detect and report AI-generated CSAM. This does absolutely nothing to help protect actual children.

estetlinus 2 hours ago

> if CSAN ever makes it into one of my systems

If that is a worry you carry around, I really hope you work harder to make it never be the case lol. FYI it’s not a normal thing to worry about.

trollbridge an hour ago

Any site that allows users to upload content has to deal with this.

RIMR 2 hours ago

If you run a website with user-generated content, you absolutely will encounter CSAM. It may not be "normal" in that it shocks the conscience, but it is absolutely normal in that every single website that accepts user-submitted content will, at some point, encounter CSAM.

It's been like that for the entire 20 years I have run these kinds of websites. It sucks, but it's just a sad reality of the world we live in.

rootusrootus an hour ago

nazgulsenpai 2 hours ago

How else do you check user generated content other than checking user generated content?

stickfigure 3 hours ago

How is this different from omni-moderation, which OpenAI already has?

https://developers.openai.com/api/docs/models/omni-moderatio...

andai 2 hours ago

>It is well established that combatting online child predators requires coordination across many organizations.

It requires all devices integrated into one single consciousness.

avazhi 3 hours ago

NCMEC is cancer.

Remember when they wanted Apple to scan everybody’s devices locally?

I’m not sure what the solution is to deal with pedophiles but I’m sure it starts with parents monitoring their kids better and keeping young children from being isolated from real world social groups. I’m also sure it doesn’t involve using classifiers on a person’s own device to detect ‘possible’ child porn, which is where this is headed. Who can forget the father who had photos of his sick baby on Google Drive for his paediatrician to see and got reported to the police.

If it really is all about protecting kids then maybe the NCMEC should be lobbying to pass laws banning kids from the internet. Think about how many harms you could reduce instantly (bye bye Instagram, Snapchat, etc).

nrmitchi 3 hours ago

Frankly, and I don't mean this in a bad way about the problem, I do not think OpenAI actually gives a shit about the underlying problem. Especially as an OpenAI-hosted-service-only, this seems more like a data grabbing play. "Scan all your images through our CSAM detector" means "send all images to us, including the 99.99% that are not CSAM!"

Don't get me wrong: these are very necessary tools for the modern internet to function, and I'm sure _drastically_ reduce the review burden on humans, but the skeptic in me does not believe that OpenAI is being entirely altruistic here.

eastbound 5 hours ago

Finally Sam Altman found something good to do in terms of my security, I'm relieved. It fits perfectly well in a world where Saudi Arabia lead the UN Commission on the Status of Women (CSW).

kittikitti 2 hours ago

This is just public relations and no reasonable person would believe OpenAI will do anything to fight against online child sexual abuse. In fact, it's much more reasonable to assume they will continue to make the problem much worse.

formerlurker 2 hours ago

I agree with you. To me, it’s all lip service and nothing gets done until people get hurt.

Similar to Roblox ignoring child grooming in their platform because they make money off of it. https://www.bloomberg.com/features/2024-roblox-pedophile-pro...

Similar to Facebook and other platforms feeding self-harm content to vulnerable children and teenagers, and even connecting them to dealers. These companies know their platforms are not safe for kids. https://www.bloomberg.com/features/cant-look-away/

bearjaws 5 hours ago

If only there was a huge data mine of child abusers sitting on some sort of email service they could go after. Maybe some sort of jeevacation inbox.

Could be a great start for "rogue AI"

Jokes aside, I'm glad they are giving tools to help fight this as currently your options are limited because every provider will simply return a content warning if you try to use an LLM as a moderator.