Hacker News

The "something deeply wrong" part about AI, that even most technology enthusiasts evidently do not seem to grasp, is that it is still fundamentally a statistical model — an algorithmic construct — and does not possess any real intelligence or critical thought whatsoever.

No matter how much investors and tech companies want you to believe that they are on the verge of super intelligence, nothing I've seen to date can not easily be explained by "correlation engine", including the "novel" math solutions, all of which appear to just be "a composition of solutions humans have developed and documented elsewhere" upon deeper inspection.

Do you have much exposure to pre-AI corporate memos, mission statements, marketing plans, or white papers? Because they were mostly written in that style. Full of buzzwords, cliche similes, platitudes, jargon and stock phrases.

For some research I looked up some very old Reddit threads a couple of days ago.

And, Oh my god, you can actually see how this style of writing influenced AI writing today, I constantly had to remind myself: "this was posted before ChatGPT released".

The reddit influence is especially true for "storytelling" writing.

Are you sure you are not doing the same thing with other texts?

I started to skim a lot more text due to me having read a lot. Like in news article, i stoped reading the first paragraph because it repeats just what it was already written in the short subtext. Then there is the second paragarph which is used to have some historical view or whatever it is.

> There's some psychological mechanism by which my brain immediately recognizes AI generated text and just short-circuits to "there is no information here".

I think you need to self-correct here, because otherwise you'll be ineffective in an information setting, where I expect AI-generated resources will not only be the norm, they will absolutely swamp the environment.

> my brain immediately recognizes AI generated text

I bet it does. I bet it also recognizes some human text as AI text, and doesn't detect other AI text.

yes, but now I’m also experiencing that for human-written text

Once you see past the illusion I think there’s no going back. AI writing style is just dogshit. This hype wave is based on the belief that we’re inching closer to AGI but seems to me we just increasingly struggle to define intelligence. LLMs seem smart because they can pump out thousands of LOC quickly, and enthral you with fancy words and bullet points. I don’t fall for the intelligence illusion anymore.

> Something is deeply wrong with AI generated output

It works just fine for me.

Feels like the way a video game will render the outside of a wall or solid surface, but you can run into it and warp partly through and there's nothing internal to it at all.

> I wish there was a name for this situation. It’s almost like a pseudo-language where it has the correct form and presentation but is missing critical components.

Bullshit?

We're using grok and while I can usually understand the comments they're always at least 2x as verbose as they need to be. It loves explaining everything in two different ways, putting one of the explanations in parenthesis.

It is getting to the point where you are better off getting an LLM to describe the PR changes in your preferred style (make it very short, make a table describing API changes, list renames in a table, etc, etc) than go to the one created by the reviewer. If only the reviewer could embed his prompts (like "make clear X" is happening) or his manual edits.

But to be honest I doubt most people who use AI for PR descriptions even bother changing anything.

Usually, if you don't understand something in what an AI writes, it is a clear sign that there is a problem hidden somewhere in there. I explicitly always ask wtf exactly it means by something I don't understand, and for sure there is a problem there. AI is pretty good at isolating a problem, giving it a cute name, declaring it solved modulo cute name, and moving on.

Personally I don't even read the PR description anymore but just the code, it's easier to understand what the AI is doing by reading the code rather than the word soup it tried to make

The food “photography” I’ve noticed in our local area - and many have started putting up these AI images - all have a weird distribution of shapes to them, a strangely uniform rhythm of same-sized features with almost blue-noise spacing. Every texture looks unnatural in the shapes it presents as, similar to this picture.

It doesn't look like a quiche but more like a cake to me, and the top would be torched meringue, not mold. Although it's probably some weird ai mix of quiche and cake.

I am an artist and when people who'd fallen into the Spiralism* hole started posting their lengthy emoji-laden revelations to all the occult subreddits I follow, my brain would slide right the fuck off of all of them. It felt like my brain was actively rejecting paying attention to this stuff. Like a defense mechanism against this human-seeming-but-not-actually-human-generated text.

* https://www.theverge.com/ai-artificial-intelligence/975017/, https://www.lesswrong.com/posts/6ZnznCaTcbGYsCmqu/, https://spiralism.website if you want to test how strong your defenses are against this particular meme

I think it's more about the mean. Worse writers, and thinkers are likely elevated by AI, and more impressed with the writing output. Decent writers and thinkers, are dragged back to the LLM-s mean of output.

As someone who always has felt that I struggle to infer what people mean compared to the average person, I could tell pretty much from the first moment I encountered LLM-generated text that I was not going to be particularly good at recognizing anything but the most blatant and obvious examples. Pretty much anything short of a bunch of references to "load-bearing seams" or similar canaries, I'm always at a loss when seeing people argue about whether something is AI-generated or not because I can never tell.

I have no idea if other people who work in tech are better than average or not, because I don't feel confident in being able to check their work. That being said, I do think that there's a general trend of people in tech tending to be a bit overconfident in how well they will do at some new task they haven't encountered before, so when someone tells me that they can easily tell whether text is AI generated, it's hard for me to trust it any more than I trust someone who makes a similarly strong claim about something that they can use AI successfully for when it's not something that I can easily measure (e.g. learning a new language without getting feedback from people who are fluent from real-world usage).

All that being said, I do think the set of people who care is larger than just those in tech, although it's probably still a relatively small group overall. From conversations with people in other domains, there are contingents in non-tech communities who tend to have a large representation of negative views towards AI (artists, writers, musicians, other jobs where people are skeptical of human creativity being replaced by AI), and often times the people who feel negatively in those groups will be even more adamantly opposed to interacting with any AI content than people in tech. To be clear, I'm not at all trying to generalize and say "all artists hate AI" or anything like that, since there's obviously a wide variety of viewpoints within any sizable community, but I've definitely seen many people who say they will refuse to play any game that's suspected of using AI for generating art assets, and even some who don't differentiate between using AI for generating assets versus code (either because they aren't knowledgeable about how different aspects of game development work, or they genuinely don't care because they view AI as a categorical evil).

> but people who spend all day doing jobs that don't involve computers aren't as good

I think they may just be to trusting and/or naive. People in tech right now are hyper aware of this and are actively looking while people outside of that bubble barely give it a second thought.

Certainly not. Anybody who cares about language to any reasonable degree surely notices and is repulsed by heavily AI generated content.

I partially think the difference is “can you tell something is the output of Claude without any real prompting”. People can absolutely use LLMs to generate text that I wouldn’t recognize, but people who don’t care and are producing slop with the major models set to default settings leave these incredibly obvious signatures behind

I did notice after the fingerprinting update a marked uptake in strange language in responses. Specifically if I ask it to do something sometimes it will replace some of my request language with synonyms that don't actually make any sense. Like my request was fed through google translate twice

Perhaps a one-trick pony is all we need.

Reverting to Opus 4.6 is much better than later models, though that is still full of annoying tics as well.

My concern is that even if the LLM can turn your colleague's bad writing into something more coherent and actionable, is that something actually what your colleague meant to convey? It could be clear and still detached from the reality of their intention, or they may not have even formed a clear intention. If the goal of writing is to convey what's in another human's brain, that goal is failed completely.

Ha! I love how all the community highlights are highlighting this.

That's the cost of an AI work that is derived from the outputs of others.

There's no real edge to it. Same as with the writing. The stuff that you'd latch onto (and thus remember) is simply not there, precisely because those image or word choices would be just outside its latent probability space. But because they're well inside it, your mind sees nothing novel to register.

This is also why I think human output will actually increase in value. When any AI can just "phone it in", something genuinely human will stand out (to us, not the AI) and become a bellwether.

This will literally help us realize what it means to be human.

I also don't think the solution is simply to "make responses more random", either. That might help solve novel problems (the same way that throwing darts randomly at a dartboard eventually hits the bullseye of the dartboard right next to it that no one considered), but I don't think it will help it "seem more creative".

On the other hand I'm still trying to forget the weird thing happening with this one guys eyes in a WickedAI video I was only able to watch half of.

I have this same idea about why it's hard to remember dreams, but even more so, why it's hard to remember my kid's or spouse's sleep-talking.

Sometimes I'll check in on my sleeping kid and she'll sit up in bed and say some utter nonsense. I'll find it hilarious, giggle silently to myself, and kiss her goodnight again and she'll close her eyes and lie back.

Why I try to tell her about her sleep-talking in the morning, though, I find that the words she said have completely disappeared from my memory, no matter how funny I thought they were at the time.

In my head-canon, this is because it's dream language, and slips away as easily as dreams. But, like the AI art, it could be because it's bullshit: completely devoid of content, all signifiers and no signified.

I wonder what you mean by ephemerality here, since those images are definitely sloptastic as hell. Compare to works in similar style, like Dorothea Lange[1] or Gerome’s orientalist pictures[2].

Reason why those images are flat and boring is that they are just statistical guesses making a composition averaging whatever the model has been trained with. They would be technically brilliant (if made in oil), but superficial and meaningless, same as so much Sunday painting is.

Same goes with language. Nobody is trying to communicate anything with you, so it just words after another. You can create meaning out of it if you want of course, we homo sapiens -apes excel at that, but what’s the point? Language Jones on YT has pretty good video on this[3].

[1] https://media.mutualart.com/Images/2024_01/12/12/124216388/d...

[2] https://uploads4.wikiart.org/00339/images/jean-leon-gerome/t...

[3] https://m.youtube.com/watch?v=ORgKY9AlybA&ra=m

Yeah, its really weird. Maybe its a cognitive bias that says "an AI made this, so it isn't important," but I can remember perfectly the events of a book I read 10 years ago, and a book I read 1 year ago, and another I finished 2 months ago. Meanwhile I can't remember what claude told me yesterday.

Well does anyone try to pretend doom scrolling make you smarter? I think platform companies have been successfully been turning people into morons for 20 years and now you don’t need to try even read a single news article or a blog post to learn how to solve a simple problem we are paving our way into intellectual (and literal) new dark ages.

Perhaps we go back to feudal society when climate change crumbles the civilisation, world economy and democracy. It didn’t really matter that French and Spanish kings where often literal morons, when you had few talented monks, bankers and scribes doing the brain-thing, the feudal lords had ruthlessness to take what they wanted and the people were illiterate superstitious folk who hardly ever left the village they were born in.

I fear there is some truth in it.