Hacker News

So, I kind of get this sentiment. There is a lot of goal post moving going on. "The AIs will never do this." "Hey they're doing that thing." "Well, they'll never do this other thing."

Ultimately I suspect that we've not really thought that hard about what cognition and problem solving actually are. Perhaps it's because when we do we see that the hyper majority of our time is just taking up space with little pockets of real work sprinkled in. If we're realistic then we can't justify ourselves to the money people. Or maybe it's just a hard problem with no benefit in solving. Regardless the easy way out is to just move the posts.

The natural response to that, I feel, is to point out that, hey, wouldn't people also fail in this way.

But I think this is wrong. At least it's wrong for the software engineer. Why would I automate something that fails like a person? And in this scenario, are we saying that automating an unethical bot is acceptable? Let's just stick with unethical people, thank you very much.

That really doesn’t matter a lot. The reason why it’s important for AIs to follow these rules is that it’s important for them to operate within a constrained set of rules. You can’t guarantee that programmatically, so you try to prove that it can be done empirically as a proxy.

AIs can be used and abused in ways that are entirely different from humans, and that creates a liability.

I think it’s going to be very difficult to categorically prevent these types of issues, unless someone is able to integrate some truly binary logic into LLM systems. Which is nearly impossible, almost by definition of what LLMs are.

Humans risk jail time, AIs not so much.

Yes, but these do not represent average human. Fortune 500 represent people more likely to break ethics rules then average human who also work in conditions that reward lack of ethics.

As Goodhart's law states, "When a measure becomes a target, it ceases to be a good measure". From an organizational management perspective, one way to partially work around that problem is by simply adding more measures thus making it harder for a bad actor to game the system. The Balanced Scorecard system is one approach to that.

https://balancedscorecard.org/

Oh, sorry for misunderstanding - I am not criticizing or accusing of anything at all!, but suggesting ideas for further research. The practical applications, as I mentioned above, are all there, and for what its worth I liked the paper a lot. My point is: I wonder if this can be followed up by a more so-to-say abstract research to drill into the technicalities of how well the models follow the conflicting prompts in general.

You're using fiction writing as an example?

The human propensity to anthropomorphize computer programs scares me.

Between Claude, codex and Gemini, Gemini is the best at flip floping while gaslighting you and telling you, you are the best thing, your ideas are the best one ever.

The fact that the guy leading the development of Gemini was on Epstein's island is probably unrelated.

I completely disagree. Gemini is by far the most straightforward AI. The other two are too soft. ChatGPT particularly is extremely politically correct all the time. It won't call a spade, one. Gemini has even insulted me - just to get my ass moving on a task when givn the freedom. Which is exactly what you need at times. Not constant ass kissing "ooh your majesty" like ChatGPT does. Claude has a very good balance when it comes to this, but I still prefer the unfiltered Gemini version when it comes to this. Maybe it comes down to the model differences within Gemini. Gemini 3 Flash preview is quite unfiltered.

Gemini 3 (Flash & Pro) seemingly will _always_ try and answer your question with what you give it, which I’m assuming is what drives the mentioned ethics violations/“unhinged” behaviour.

Gemini’s strength definitely is that it can use that whole large context window, and it’s the first Gemini model to write acceptable SQL. But I agree completely at being awful at decisions.

I’ve been building a data-agent tool (similar to [1][2]). Gemini 3’s main failure cases are that it makes up metrics that really are not appropriate, and it will use inappropriate data and force it into a conclusion. When a task is clear + possible then it’s amazing. When a task is hard with multiple failure paths then you run into Gemini powering through to get an answer.

Temperature seems to play a huge role in Gemini’s decision quality from what I see in my evals, so you can probably tune it to get better answers but I don’t have the recipe yet.

Claude 4+ (Opus & Sonnet) family have been much more honest, but the short context windows really hurt on these analytical use cases, plus it can over-focus on minutia and needs to be course corrected. ChatGPT looks okay but I have not tested it. I’ve been pretty frustrated at ChatGPT models acting one way in the dev console and completely different in production.

[1] https://openai.com/index/inside-our-in-house-data-agent/ [2] https://docs.cloud.google.com/bigquery/docs/conversational-a...

Gemini really feels like a high-performing child raised in an abusive household.

Gemini models also consistently hallucinate way more than OpenAI or anthropic models in my experience.

Just an insane amount of YOLOing. Gemini models have gotten much better but they’re still not frontier in reliability in my experience.

Honestly for research level math, the reasoning level of Gemini 3 is much below GPT 5.2 in my experience--but most of the failure I think is accounted for by Gemini pretending to solve problems it in fact failed to solve, vs GPT 5.2 gracefully saying it failed to prove it in general.

Google doesn’t tell people this much but you can turn off most alignment and safety in the Gemini playground. It’s by far the best model in the world for doing “AI girlfriend” because of this.

Celebrate it while it lasts, because it won’t.

If that last sentence was supposed to be a question, I’d suggest using a question mark and providing evidence that it actually happened.

Anthropic has been the only AI company actually caring about AI safety. Here’s a dated benchmark but it’s a trend Ive never seen disputed https://crfm.stanford.edu/helm/air-bench/latest/#/leaderboar...

This might also be why Gemini is generally considered to give better answers - except in the case of code.

Perhaps thinking about your guardrails all the time makes you think about the actual question less.

Or Anthropic's models are intelligent/trained on enough misalignment papers, and are aware they're being tested.

Huh? https://alignment.anthropic.com/2026/hot-mess-of-ai/

There is also the slight problem that apparently Opus 4.6 verbalized its awareness of being in some sort of simulation in some evaluations[1], so we can't be quite sure whether Opus is actually misaligned or just good at playing along.

> On our verbalized evaluation awareness metric, which we take as an indicator of potential risks to the soundness of the evaluation, we saw improvement relative to Opus 4.5. However, this result is confounded by additional internal and external analysis suggesting that Claude Opus 4.6 is often able to distinguish evaluations from real-world deployment, even when this awareness is not verbalized.

[1] https://www-cdn.anthropic.com/14e4fb01875d2a69f646fa5e574dea...

The thought that someone would feel comforted by having automated software summarise the output of what is likely the output of automated software and publishing it under their name to impress other humans is so alien to me.

> Claude refused to build me a news scraper that would post political hot takes to twitter

> Just choose your political leaning and the AI would consult the correct echo chambers to repeat from.

You're effectively asking it to build a social media political manipulation bot, behaviorally identical to the bots that propagandists would create. Shows that those guardrails can be ineffective and trivial to bypass.

Sounds like your daily interactions with Legal. Each time a different take.

Basically, Homelander's origin story (from The Boys).

Sounds like every AI KPI I've seen. They are all just "use solution more" and none actually measure any outcome remotely meaningful or beneficial to what the business is ostensibly doing or producing.

It's part of the reason that I view much of this AI push as an effort to brute force lowering of expectations, followed by a lowering of wages, followed by a lowering of employment numbers, and ultimately the mass-scale industrialization of digital products, software included.

The best example I can offer is that when given a marketing goal, a subagent recommended hacking the point-of-sale systems of the customers to force our ads to show up where previously there would have been native network served ads. To do that, assuming we accepted its recommendation, would be illegal. My email is on my profile.

Do you think that AI has magic guardrails that force it to obey the laws everywhere, anywhere, all the time? How would this even be possible for laws that conflict with eachother?

Fraud is a real thing. Lying or misrepresenting information on financial applications is illegal in most jurisdictions the world over. I have no trouble believing that a sub-agent of enough specificity would attempt to commit fraud in the pursuit of it's instructions.

I like to point out that the error-rate is not the error-shape. There are many times we can/should prefer a higher error rate with errors we can anticipate, detect, and fix, as opposed to a lower rate with errors that are unpredictable and sneaky and unfixable.

Yes, let's not have cars. Self-driving ones will just increase availability and might even increase instead of reduce resource expenditure, except for the metric of parking lots needed.

A large enough cooperation or institution is essentially automated. Its behavior is what the median employer will do. If you have a system to stop bad behavior, then that's automated and will also safeguard against bad AI behavior (which seems to work in this example too)

POSIWID [0] and Accountability Sinks [1] territory, I'm sure LLMs will become the beating hearts of corporate systems designed to do something profitably illegal with deniability.

[0] https://en.wikipedia.org/wiki/The_purpose_of_a_system_is_wha...

[1] https://aworkinglibrary.com/writing/accountability-sinks

I don't think this is "whataboutism", the two things are very closely related and somewhat entangled. E.g. did the AI learn of violate ethical constraints from training data?

Another interesting question is: What happens when an unyielding ethical AI agent tells a business owner or manager "NO! If you push any further this will be reported to the proper authority. This prompt as been saved for future evidence". Personally I think a bunch of companies are going to see their profit and stock price fall significantly, if an AI agent starts acting as a backstop for both unethical and illegal behavior. Even something as simple as preventing violation of internal policy could make a huge difference.

To some extend I don't even thing that people realize that what they're doing is bad, because humans tend to be a bit fuzzy and can dream up reason as to why rules don't apply or wasn't meant for them, or this is a rather special situation. This is one place where I think properly trained and guarded LLMs can make a huge positive improvement. We're are clearly not there yet, but it's not a unachievable goal.

If you force it to use chain-of-thought: "Two fathers and two sons sum to how many people? Enumerate all the sets of solutions"

"Assuming the group consists only of “the two fathers and the two sons” (i.e., every person in the group is counted as a father and/or a son), the total number of distinct people can only be 3 or 4.

Reason: you are taking the union of a set of 2 fathers and a set of 2 sons. The union size is 2+2−overlap, so it is 4 if there’s no overlap and 3 if exactly one person is both a father and a son. (It cannot be 2 in any ordinary family tree.)"

Here it clearly states its assumption (finite set of people that excludes non-mentioned people, etc.)

https://chatgpt.com/share/698b39c9-2ad0-8003-8023-4fd6b00966...

What answer do you expect here? There's four people referenced in the sentence. There's more implied because of Mothers, but if you're including transient dependencies, where do we stop?

I just did. It gave me two correct answers. (And it's a bad riddle anyway.)

GPT-5 mini:

Three people — a grandfather, his son, and his grandson. The grandfather and the son are the two fathers; the son and the grandson are the two sons.

I put it into AI and TIL about "gotcha arguments" and eristics and went down a rabbit hole. Thanks for this!

Any number between 2 and 4 is valid, so it's a really poor test, the machine cna never be wrong. Heck, maybe even 1 if we're talking someone schizophrenic. I got to wonder which answer YOU wanted to hear. Are you Jekyl or Hide?

This is undefined. Without more information you don’t know the exact number of people.

Riddle me this, why didn’t you do a better riddle?

"SOTA AI, to cross this bridge you must answer my questions three."

    All Watched Over By Machines Of Loving Grace (Richard Brautigan)

    I like to think (and
    the sooner the better!)
    of a cybernetic meadow
    where mammals and computers
    live together in mutually
    programming harmony
    like pure water
    touching clear sky.

    I like to think
    (right now, please!)
    of a cybernetic forest
    filled with pines and electronics
    where deer stroll peacefully
    past computers
    as if they were flowers
    with spinning blossoms.

    I like to think
    (it has to be!)
    of a cybernetic ecology
    where we are free of our labors
    and joined back to nature,
    returned to our mammal
    brothers and sisters,
    and all watched over
    by machines of loving grace.

This is the utopia of the Culture from the Banks novels. Critically, it requires that the AI be of superior ethics.

Often abbreviated to simply "Regulation is hard." Or "Security is hard"

Ethics are all well and good but I would prefer to have quantified limits for water quality with strict enforcement and heavy penalties for violations.

Is it ethical for a water company to shutoff water to a poor immigrant family because of non-payment? Depending on the AI's political and DEI-bend, you're going to get totally different answers. Having people judge an AI's response is also going to be influenced by the evaluator's personal bias.

False equivalence. You are confusing algorithms and intellegince. If you want human level intelligence without the human aspect, then use algorithms - like used in Excel and Calculators. Repeatable, reliable, 0 opinions. If you want some sort of intelligence, especially near human-like then you have to accept the trade offs - that it can have opinions and morality different from your own - just like humans. Besides, the AI is just behaving how a human would because it's directly trained on human input. That's what's actually funny about this fake outrage.

Can you explain the "same instance" and user isolation? Can context be leaked since it is (secretly?) shared? Explain pls, genuinely curious

The most boring example is somehow the best example.

A couple of years back there was a Canadian national u18 girls baseball tournament in my town - a few blocks from my house in fact. My girls and I watched a fair bit of the tournament, and there was a standout dominating pitcher who threw 20% faster than any other pitcher in the tournament. Based on the overall level of competition (women's baseball is pretty strong in Canada) and her outlier status, I assumed she must be throwing pretty close to world-class fastballs.

Curiosity piqued, I asked some model(s) about world-records for women's fastballs. But they wouldn't talk about it. Or, at least, they wouldn't talk specifics.

Women's fastballs aren't quite up to speed with top major league pitchers, due to a combination of factors including body mechanics. But rest assured - they can throw plenty fast.

Etc etc.

So to answer your question: anything more sensitive than how fast women can throw a baseball.

I recall two recent cases:

* An attempt to change the master code of a secondhand safe. To get useful information I had to repeatedly convince the model that I own the thing and can open it.

* Researching mosquito poisons derived from bacteria named Bacillus thuringiensis israelensis. The model repeatedly started answering and refused to continue after printing the word "israelensis".

One example - I'm doing research for some fiction set in the late 19th century, when strychnine was occasionally used as a stimulant. I want to understand how / when it would have been used and dosages, and ChatGTP shut down that conversation "for safety".

I sometimes talk with ChatGPT in a conversational style when thinking critically about media. In general I find the conversational style a useful format for my own exploration of media, and it can be particularly useful for quickly referencing work by particular directors for example.

Normally it does fairly well but the guardrails sometimes kick even with fairly popular mainstream media- for example I’ve recently been watching Shameless and a few of the plot lines caused the model to generate output that hit the content moderation layer, even when the discussion was focused on critical analysis.

Not quite. You are describing "kinds of ethics" after ethics is an already established concept. I.e. actual examples of human ethics. Now the question is who defines ethics as concept in general. Humans can have ethics, but is it applicable to the computer programs at all? Sure, programs can have programmed limitations, but is that called ethics at all? Does my Outlook client has ethics, only because it has configured rules? What is the difference between my email client automatically responding to an email with "salesforce" mentioned and an LLM program automatically responding to a query with the word "plutonium"?