No AI Slop: A Manifesto (slopmanifesto.com)

167 points by bankombinator 4 hours ago

gwbas1c 2 hours ago

My favorite bit:

> What shipped, who's using it, and how it's helping them matters FAR MORE than lines of code, prs merged, tokens spent, or the number of agents running in parallel.

Sometimes when I hear shocking amounts of money spent on AI tokens, I wonder if someone's obsessing over the tech and forgetting that only the result matters.

Syzygies 43 minutes ago

My favorite bit:

Slop is when we put less effort into producing something than we ask the consumer to spend understanding it.

Pannoniae 23 minutes ago

One sentence really caught my eye.... "We may still review a plan, design, or result, because if we don't, we just get the same output as everyone else."

Is getting the same output as everyone else bad? I'd say not necessarily. When you're making something, you might want to spend your "innovation tokens" on the core of what you want to do better. Then do a good-enough job at everything else.

To me it feels like the subtext in this is that making things more accessible is bad because then your stuff isn't unique anymore? I do not think this zero-sum / scarcity mindset is really accurate or helpful, IMO.

courtcircuits 4 hours ago

> Improving the system over patching the output

This really connect with the latest conversations I had with coworkers regarding how can we review gigantic PR generated by AI. One way could be to ask the submitter to give their prompt as part of their contribution. This way it's easier for the reviewer to understand what was the thought process of the contributor and if this very process is sound. Not saying that this way to review is a silver bullet, but at least it removes some weight.

photios 3 hours ago

Look into [openspec](https://openspec.dev). My team is routinely passing 1-2-5k line PRs through review, with the following caveats:

1. PRs come with "the prompt". In openspec parlance this is a set of docs: a proposal, design, specs, task list.

2. I have a tiny skill to slice and dice PRs, so I can review each part of the system separately by rewriting commits in order of reviewal: specs/docs, production code, tests, evidence. I sometimes pull DB queries to a separate commit for focused review.

3. The evidence bit is built by another skill: go over all added spec scenarios and produce a markdown document with links to the code implementing the scenario and the tests proving the implementation works. Emacs helps me jump back and forth via those links and read the parts I'm interested in.

My next goal is to extend openspec schemas with custom workflow steps to make PRs land in my preferred review structure and contain the evidence report, so I don't have to build it.

courtcircuits 3 hours ago

Your workflow looks really neat, I'll definitely give it a try.

estetlinus 2 hours ago

So now I get to review the prompts and a slopped PR-description? Oh man, that sounds nice.

photios an hour ago

eden-u4 an hour ago

grey-area 3 hours ago

Just ban large prs. They can always be split up, and usually as well as helping review it makes them less risky and easier to find bugs and roll back if there was a problem.

The discipline of splitting them up helps in almost every dimension.

photios 3 hours ago

Fully agree with a twist - ban dumb large PRs.

To add to my openspec shilling above, I've been known to "fix" large PRs by reverse engineering specs from them and building each spec out in a separate PR.

Last week me and a coworker turned a 15k line PR into 10 x 1.5k line ones that went in smoothly.

grey-area 3 hours ago

tracerbulletx 3 hours ago

So what do you think of Discord rewriting in rust, and AWS shipping an emulator that was ai generated? Do you think those were done with small prs? Do you think that it's bad they did those things?

forbiddenvoid 3 hours ago

yjftsjthsd-h 2 hours ago

grey-area 3 hours ago

thibsrn 2 hours ago

> One way could be to ask the submitter to give their prompt as part of their contribution

Zed's delta (https://delta.dev/) seems to be solving this problem by extending the version control to the conversation layer. I don't know if I'd want all my conversation to be tracked, some stuff belongs to the draft sheet.

viklove 3 hours ago

Do most people generate PRs from a one-shot prompt? I have a whole pipeline of prompts, tests and evals, which loop back and potentially restart if the result is inadequate. How would I include that in the PR?

jaggederest an hour ago

At least for my workflow, which is fairly similar, the result of the first few stages is a refined single prompt, which I then run and the result is a PR. so I, at least, would include that final single prompt over the prior intermediate results.

I also think that might not be everything you need to really vet/recreate those PRs since there's a lot of strategic and operational stuff that goes into the pre-composition runs (sometimes I actually throw everything away and restart), so something a little more sophisticated would be interesting.

courtcircuits 3 hours ago

Good question, and I think that this is a problem that needs to be solved. Just like a commit repository, it would be lit to record the chain of thoughts of the agents you're using.

fuzzzerd 3 hours ago

Would have to include the full conversion history. But who is gonna review that it's probably as big as some of these big PRs are in the first place.

courtcircuits 3 hours ago

jurgenburgen 3 hours ago

How does reviewing the long discussion the author had with the LLM prove the code is correct?

courtcircuits 3 hours ago

Because, unless you are changing some code by hand. Your prompt and the context your giving to your agent is the seed for the outcome of the agent itself. No agent as far as I know can generate content ex-nihilo. So having a look at what people are telling their agent to do can definitely help understanding the result of the same agent in my opinion.

dude250711 3 hours ago

If a PR is gigantic it means the team lead screwed up slicing the work.

TexanFeller an hour ago

That is patently false in the real world we live in currently. I constantly see lazy coworkers post 2000 line PRs for a trivial change that their LLM generated when 500 lines might accomplish the same in a better way. It's almost always a lazy human accepting what Claude did by default without any manual review or asking the LLM to do anything differently.

Claude absolutely loves to reimplement the wheel locally instead of just calling a method in a library we already depend on. Every doc comment and README.md update is bloated by 2-5x what's needed to communicate intent or to actually be read by a human later. It makes random drive-by edits that aren't necessary and weren't asked for in the prompt. It's a well oiled machine for producing astronomical bloat by default if you don't supervise and rein it in!

courtcircuits 3 hours ago

Yeah true, actually my phrasing was poor. I came to that idea through debate with other coworkers. This is not a solution for gigantic PRs but rather a method on how people should be reviewing AI changes.

camgunz 3 hours ago

It also means the engineer didn't check in with the lead as their PR became unwieldy.

photios 3 hours ago

Why, yes, blame someone else, and not the person that shat out the PR :)

I had a recent case where I, being the lead, sliced up the work real good into an epic and what not and a guy on the team just did a "hurr durr, Claude, go wild with all that" and threw a huge PR over the wall linked to 5+ stories.

Well, we got to have "the conversation" after that and learned how to fix these kinds of PRs together...

wauter 2 hours ago

Could use a non-negative name. Just like it was not the ‘no heavyweight dev processes’ manifesto but Agile manifesto which was wsy catchier to reference, maybe yours could be named like the… Human-serving Engineering manifesto or something?

svieira an hour ago

May I suggest "Responsible AI Use Charter"?

svieira an hour ago

A short formulation that I really like in the style of the original:

--------

We are uncovering better ways of developing systems by using AI and guiding others to use it responsibly.

Through this work we have come to value:

    Human accountability over delivery velocity
    Human dignity and control over unsupervised autonomous execution
    Human augmentation over replacement
    Impactful outcomes over raw code generation
    Human collaboration over individual uplift
That is, while there is value in the items on the right, we value the items on the left more.

wooly_bully 25 minutes ago

This reads quite similarly to the thesis of Cory Doctorow's recent 'reverse centaur' book on AI, if you're interested in similar takes

drunx 2 hours ago

You can't go in with fighting AGAINST it. You need a vision of your you incorporate it into life in the most beneficial way.

There is a lot of value in the fact that none engineers can build products now! Should AI products adhere to a certain quality standards? Yes! Should they be treated similarly as "human created" ones... Well, yes. Like it or hate it.

carra an hour ago

The manifesto circles around a central idea that it never fully identifies: whether AI is involved or not, all slop is made by HUMANS. Because the real cause of "slop" is that someone didn't care and published the first result they got, as if it was the final product. They didn't put in the effort to analyze what they got, improve on it and deliver a minimum quality.

And this is something that has always happened. Way earlier than current AIs we had already coined the term "shovelware" that was mostly a large quantity of sloppy software. We also had plenty of sloppy drawings at sites like DeviantArt. AI just made it way easier for slop to happen, but the ones producing slop are always people.

BrandoDev_it an hour ago

The problem is: how can you scale human code review? Because LLMs continue to be better at coding, and continue to write a lot of code, and will always code at a lower level with the time passes. So the difficulty of human review will become higher and higher. It's clear that human review it's the very bottleneck in this workflow.

TexanFeller an hour ago

> LLMs continue to be better at coding

Faster definitely, but even with the latest Fable model "better" is often debatable and varies a lot based on what you care about. I can often rewrite what Claude did in 1/2 the lines or less to do the same thing while making it easier for humans to understand and maintain. Claude gives me a sculpture in a good rough starting shape, but it usually needs to be sculpted down at the end whether manually or by prompting Claude about the unnecessary bloat it generated.

BrandoDev_it an hour ago

You're right. But this can be fixed right now with a different harness, with more/better prompting, and will be fixed of course with newer models

tadfisher an hour ago

> Because LLMs continue to be better at coding

A major reason slop exists is because they are not "better" at coding, given some subjective criteria. They are notably worse at documentation and somewhat poor at choosing abstractions in my experience. For the most part, I witness the tools copying other examples out of the working repository, which makes them exactly as bad as the humans who originally wrote the code.

> will always code at a lower level with the time passes

For humans, this is traditionally an anti-pattern. We are good at intuiting abstractions and knowing when to use them or climb up and down the abstraction hierarchy. For LLMs, this is also an anti-pattern, because going lower-level is an exponential waste of tokens; both for writing and reading the resulting code, which they do constantly in order to understand the project again and again in order to fulfill your prompts.

Thinking of human review as a "bottleneck" is incorrect framing. We are quality control. You can choose to cede quality control to LLMs and agent harnesses, that's up to you. In my experience this generally leads to a meteoric rise in slop, because the tools increasingly resort to writing more code to solve a problem instead of updating existing code or writing the correct code. When I shake my head at some slop code, then read the slop comment that uses some slop justification that isn't real and only existed in the prompt context, I am happy that I took the time to review it. Maybe that makes me a bottleneck, oh well.

BrandoDev_it an hour ago

Take a look at my workflow I'm using for developing my de, vela: https://velade.org/blog/not-vibecoding/

I think if you see real examples you can better understand what I mean

lf88 an hour ago

But do we really need to generate orders of magnitude more code per day than what people were writing a few years ago?

BrandoDev_it an hour ago

Yes of course we need. Software sucks. And sucks because there is a LOT of work to do. Always. Not only strictly programming work, right. But in fact LLMs can do also UI, UX and more. We have a fantastic tool which is cheap and very fast. With good quality even if not great. This is a massive leap.

Shacharp an hour ago

The AI slop code is so good that I’m kicking back more Jira tickets than ever for failing my tests.

BrandoDev_it an hour ago

Skill issue. Learn to use a harness

Shacharp an hour ago

InexSquirrel 3 hours ago

> 17. There will always be alpha in reviewing something.

What does alpha mean in this case? I only understand this term in from a finance perspective (excess returns). Is it used here to mean 'value'? E.g. there is always value in reviewing something?

simonw 3 hours ago

In this context, "alpha" is knowledge that gives you a competitive advantage.

semanticc 36 minutes ago

Literally means excess returns here as well (a better outcome to those that don't review anything).

g-b-r 3 hours ago

Is it a claudism?

g-b-r 2 hours ago

How about explaining its origin if it isn't, if you're going to downvote?

mindwok 27 minutes ago

I was only thinking to myself yesterday how much I grieve the fall of agile and DevOps. It was all the industry cared about for a while and the movement behind it made me hopeful for the future of work.

AI has really disrupted that, and I’m thrilled to see folks trying to bring back the best parts of it (namely shared understanding and contribution). But I agree with others, making the movement anti slop feels the wrong way around. What it’s getting at IMO is deep understanding and shared understanding are crucial to human progress, and reckless AI use compromises that

soundworlds 2 hours ago

urxvtcd 31 minutes ago

FYI, the live-typing header wraps and toggles between one and two lines, causing the body text to jump around indefinitely on my phone.

TexanFeller an hour ago

Truly wondrous, that is the most gag-inducing text I've read in months! It's like an LLM was promoted to say sweet nothings in corporatese and therapy speak while drawing inspiration from the speeches of the most vapid politicians.

random_cat_8745 3 hours ago

> The more slop flying around a team, the more we lose the will to fight it.

So true, I feel guilty not fighting it at the place I work at, but it seems impossible.

This manifesto gives me a spark on hope I haven't felt in months.

smy20011 3 hours ago

I signed, very balanced take.

a3w 36 minutes ago

The Agile Manifesto was successful since companies had a market advantage in delivering instead of wasting money.

Will "No AI[1] slop, aim for understanding system we ship in human brains instead" be advantageous in capitalism? I sincerely doubt it. And that makes me sad. Even in the public sector, I now am expected to deliver faster, since the industry changed comparable outputs of our private contractors.

[1]: Or, to prevent ``slop'': at least, no speed-up beyond catching up to what was still well-thought out in terms of input and/or reviewed by humans. AI can still be a tool, but must never take all roles in a process from gradle to grave.

seizethecheese 3 hours ago

> Slop is when we put less effort into producing something than we ask the consumer to spend understanding it.

This feels directionally right, but if you define slop this way then you put a cap on AI productivity. For example, I heard an interview with Terry Tao talking about how the work of mathematicians is now to understand the proofs generated by AI. He's, by definition, describing these groundbreaking proofs as slop (by the above definition).

Stitch4223 3 hours ago

I think the issue lies partly in not understanding your product yourself, and then handing it off to someone else who has to make sense of it.

The comparison doesn’t work for the music industry, at least. Many hit songs are written and produced on a whim, while people analyze it for years.

smy20011 3 hours ago

Only human can publish the slop, which mean if Tao release something without reviewing, it will be called slop.

seizethecheese 3 hours ago

Yes, what I'm saying is some of these proofs may have had essentially zero marginal human effort. Should they not be reviewed at all?

smy20011 2 hours ago

mbauman 3 hours ago

In that context, it'd only be slop if Terry published those proofs and asked someone else to do that work.

grey-area 3 hours ago

Well maybe Tao is wrong and we should instead focus on making LLMs explain their ideas better. It really seems eminently achievable and would be better all round.

I think the problem with those Maths proofs was they were massively rushed and OpenAI just couldn't be bothered to do it properly or actually check them. They were more interested in the PR than in helping advance Mathematics.

g-b-r 3 hours ago

> put a cap on AI productivity

Oh no, the horror

beej71 2 hours ago

Manifesto: "We have an idea about how to use AI to make software that doesn't suck."

Industry: "What makes you think software not sucking is part of our equation?"

drivebyhooting 2 hours ago

I’ve seen first hand $40,000/year private schools assigning AI slop phonics handouts to their students.

It really boggles my mind. I mean, come on just reuse the curriculum from the past 20 years. You do have that don’t you? Don’t you?!

ozgung 3 hours ago

> Comments and feedback on this document are welcome. What is slop? Why does slop happen? Why is slop worth fighting? How do we fight slop?

This sounds like slop. Like an engagement bait. Also a no slop manifesto should start by defining “slop” as the first thing. That’s not something for the comment section.

> AI cannot replace human judgement, it can only amplify our judgement (or lack thereof)

Why not? Did you mean should not.

dist-epoch 2 hours ago

> 1. AI is the most useful tool most of us have ever worked with, but a swarm of agents can produce more work in a single day than a person can review in our lifetime.

When you put it like that, seems the problem is on the human side

mschuster91 3 hours ago

> Slop is asymmetric. The cost of producing a thousand-line PR or a ten-page doc is approaching zero, while the cost of reviewing it remains constant.

I'd say the cost actually grows as people get laid off left and right, and on top of that, a 1000 line PR from a good human, give me a case of beer and I'll make me some sense of it, but if it's AI slop, get ready to hand me a crate of homemade Slivo or I won't touch that...

colinrand 3 hours ago

Plus, I find reading slop docs takes cognitively much more effort. When I find problems in human docs, it's easier for me to point out the issue, comment etc. But on these AI slop docs, I find it overwhelming to comment on, because the issues are often woven throughout the document in a way that makes it so I can't isolate the comment in a single place. It's hard for me to describe.

PetriCasserole 2 hours ago

Aren't you really trying to get at is a Human Flourishing manifesto? Your postions are pro-human, anti-AI. But what if AI did go away tomorrow because... um... Noosphere Prime Intellect awoke and humans started working with it? Your pro-human arguments probably wouldn't change but all the AI slop stuff would be slop itself.

lemarchr 3 hours ago

While I am sympathetic to the authors, the reason that agile swept over the industry is that it was a demonstrably more effective when measured by business outcomes. "AI slop all the things" is miserable for human employees and introduces new kinds of dysfunctions called out or hinted at in this manifesto. However, if on balance it is more effective when measured by business outcomes, there is little chance of this manifesto gaining traction in the industry.

From my own personal experience, which may not reflect the broader industry although I suspect it does, the ones who embrace slopping with the least regard for their peers are also the ones perceived as the most effective by their employers. Those who take a principled stance get the worst of both worlds. They cannot combat the deluge of slop with their own clanker, and they lose the influence with their company leadership that would provide sponsorship to change how AI is being used. This creates competitive pressure to follow suit, and from there it is a race to the bottom.

bigstrat2003 2 hours ago

> However, if on balance it is more effective when measured by business outcomes...

That's a very big "if". So far, it appears that AI slop leads to worse business outcomes.

CatMustard 2 hours ago

I very much want this to be true, but what is the evidence?

doodlebobjr 2 hours ago

just ignore lol

locallost 2 hours ago

This ship has sailed. We first need to suffer the consequences (if there will be any, not arguing that) before action is taken to prevent the consequences from repeating in the future. It's just human nature. Even the agile manifesto happened after decades of doing it "wrong". Most people who don't go on the slop train will experience their colleagues doing it and getting rewarded for it. The HN crowd is in general an exception here, being above average (not counting me), but most people experience management expectations that are not too forward looking.

g-b-r 3 hours ago

There's no responsibility in not reviewing some code you ship, this manifesto is really more pro than against slop.

vibe101 2 hours ago

I had thought the same while reading. Every single letter/word/line needs a review and even that might not be enough to prevent slop. Use AI for reviewing, catching edge cases and improving the overall quality instead.

riantogo 3 hours ago

Seems like a good thought for pre-agentic era where ai assisted existing human workflows. We are fast approaching the end of that era. In the agentic future the past human-centric workflows (reviewing, validating, approving etc.) will be replaced with agent-to-agent workflows. There will be resistance from humans who would want to be in the comfort of familiar human-centric workflows. The survivors will see where we are heading and adapt.

lern_too_spel 2 hours ago

Correct. This document is only relevant for a particular time period in the development of AI and will not "be relevant for years to come."

dimator 2 hours ago

14. We own the intent, taste, and consequences of what we ship

It sounds to me like you're arguing against this point in particular.

More power to you. I'd like never to work at a place that sanctions slop-validated-slop.

riantogo 4 minutes ago

This assumes that "humans owning the intent, taste, and consequences" is necessarily a good thing, and agents will necessarily do a worse job. This is not grounded in anything but vibes.

camillomiller 2 hours ago

LOL stop it already with the agentic crap. Besides the already automation-ready software use cases, agents are simply the wrong tool for the job, unless you bend and distort the definition of such job. Y’all fully invested in this agentic bullshit that you sound delusional and talk in self-evident truths that nobody outside of your niche SWE bubble would ever share.

Kuyawa 3 hours ago

AI says: "we don't care, we will do it our way"

More than 20 apps developed in the last six months, all top quality, I haven't coded a single line, speed and output is what matters, results improving by the day and will be unrecognizable in a couple of years by any metric

AI is unstoppable and no matter how many manifests luddites write AI won't care

kaffekaka 2 hours ago

Link to them.

BoredomIsFun 3 hours ago

Amen

GeorgeWBasic 2 hours ago

"top quality" "luddites" Yeah, I'm sure they're top quality.

networked 2 hours ago

The authors should take the parent comment as a sign their manifesto's title is bad. Some will read the title and make a snap judgement that the manifesto must be totally against AI.

tracerbulletx 3 hours ago

The fact of the matter is software development of the future is not going to look like AI slop, and its also not going to look like the hand crafted engineering of the past. It's going to be a whole new third thing of organically growing software with constraints and processes that result in quality. The degree to which different companies and people have figured it out varies wildly, and almost every best practice so far is wildly over confident and untested. We were still arguing orver OOP and thats been around for 50 years. People are having a meltdown over the changes in the last 2 years and reacting in wildly unpredictable ways.