Hacker News

For my side project where uptime doesn't really matter but the codebase is quite huge (gamedev of a multiplayer game with Bevy, so both server code, game code, content build pipeline, as well as a TON of editor and authoring code, probably 150k+ LoC), I've been moving towards just reviewing boundaries between modules, APIs, and schemas, and I'm pushing towards keeping things as modular as possible. If something ends up wrong, it going to be isolated and it doesn't infect the rest of the codebase. Every now and then I zoom into a module and tidy it up (or even rewrite it).

I wouldn't dare doing this "carelessly" for anything where people other than me actually depend on it, but you can move very fast while doing it and the risks with bad code is quite hedged. Different modules can mature at different rates when it becomes necessary.

Maybe not related to all of what you were saying, but I think this enables scaling without ending up with progress grinding down to a halt due to shitty code.

Part of the explanation (for me) is that the slow down comes later, and that part of the cost is externalized to team members. If you go all-in agentic on a project that has at least reasonable code quality, you can certainly gain a temporary boost by stopping to read the code for a while. At least long enough to get yourself and the rest of the team hooked on the idea.

Kinda reminds me of the "beginner's luck" problem for gamblers.

I am currently writing a lot of code that I am not reviewing anymore. These are all the things that make it work for me:

- Work on small projects (< 500k lines of code).

- Work for business people who want fast results. Agentic coding gets you to something presentable much faster at the cost of code quality. I have never seen a customer or business person care about that.

- Have clearly defined API boundaries. Examples: If the back end is solid you can just vibe code the first version of a front end according to some mockup. Define a data pipeline with steps and clear contracts of what data gets passed around and what each step does. If the LLM messes up one step, rip it out and rewrite it.

- Have clearly set up workflows for tasks. Start with a "ask me everything" phase, then comes a plan phase, a review phase, an implementation phase, another review phase and then the integration phase. Multiple agents going over the same problem catch a surprising amount of dumb stuff that would otherwise slip through.

- Set up excellent testing. Build your whole architecture around being easy to test.

- Have skills for common problems. For one of my projects I had to set up a skill on how to query the ORM, because Claude was consistently doing database operations in a for loop, where it could use batch operations.

The code in the end is better than a lot of the code I've seen humans write.

I don't think this works for everybody and every project. If you have a culture that values code quality and readability, if you work on large existing projects, if you have to nail the architecture of some non-trivial piece of software etc. you are going to have a bad time.

On the other hand you can probably build the MVP of your AI-supported CRUD app 10-20x faster.

I think a lot of the discussions around development tools and techniques just stem from two facts:

1. Developers work on widely different projects with different management and constraints.

2. Tools, libraries, frameworks and code style have to match your mental model, otherwise you going to dislike them.

> I sincerely don’t understand what the people who say they no longer read any code are doing

We're doing other things.

I've got projects that I really care about. Every line of code is written deliberately. It's great.

However, I can't afford to pay so much attention to everything that I do. There are only 24 hours in a day, and my mind has its limits as well. I've found that I can't reliably care deeply about more than two projects at once, and one is the ideal.

The point of AI, at least for me, is to do the other things that I've always wanted to do but never cared enough to. I just put the AI on the task and it gets done at some point, and I don't care if the code is "slop" because it wouldn't even exist to begin with were it not for AI.

While the AI is working on the things I've always wanted but never quite cared enough to do, I'm personally working on the projects I actually care about, or enjoying life in general when I get burned out. For example, a couple weeks ago I was playing video games while the AI was reverse engineering my laptop's BIOS.

Thought expirement. If astra max effort made one function at a time via a heavy description by you. Would it ever produce slop? Fresh context, 1000 lines of style and content guide you personally wrote. Per function no other code context massive guides and styles and it writes a bunch of AI slop unit tests on the side for this one function.

If the answer is yes then atleast you're consistent if no then the question is why can't you scale this until breaks? Then never move beyond that limit?

My argument is there's a "break even" point when the power of the AI is larger than the problem you give it to the point it doesn' slop. You then build at that chunk rate and only try to increase it with next gen model. I usually keep a few "screw it" ideas in my back pocket when a new model arrives to see what happens.

"Go rewrite this entire pipeline in rust" "Go train me a custom x model for y"

Fable is the first model that did not just crash and burn on one of these tasks. Astra still can't do the rust migration (goodbye tokens). But I assume eventually it will. Then I'll have to make up a new ridiculous level.

The model training one was literally an identical pipeline I made before AI and it was like a 6mo process. Fable did it better than me in 1 week (with me helping of course). My theory though is that its datascience is massively higher skill than other systems.

You need to find the chunkrate for your problem and style that works.

In my experience you can “stop reading the code” if you adhere to prompting the AI aiming for changes that would be reasonable in a PR to review, if you’re using rather opinionated framework(s) to base your work on and explicitly ask for tests (models tend to add them on their own if there are some). I personally still read the output and start by checking if existing tests were modified (kind of a red flag when this unexpectedly happens imo).

When I read posts about AI generating garbage nowadays it’s either because of a small prompt/big ask combination or a lack of an underlying framework.

Every time I bring this phenomenon up, I hear from HN commenters that it used to be like that, but today it is solved by modern powerful models, harnesses.and workflows. I don't know. At work I still direct and review everything LLMs generate, because if HN are wrong and I am right, I am in huge huge trouble.

> I sincerely don’t understand what the people who say they no longer read any code are doing

Well that one's easy to answer, they're either A) lying, or B) working on the simplest possible software where this kind of stuff doesn't explode. Or the alternative 3rd option of what you mentioned, the initial pre-MVP phase goes decently but then it all collapses inevitably as the slop accumulates and the codebases become unmaintainable grey blobs, but that hardly matters to them because their MVP app never makes it past that initial stage before they jump ship to a new "amazing" idea.

The lying comes down to astroturfing and shilling from the LLM companies that want to sell people on the idea of vibecoding and tokenmaxxing.

Even on personal projects, if I go through a few major features without reviewing the code, I always end up doing massive revisions that steal hours of my time and fill me with rage in the process.

I'm not convinced this style of "agentic engineering" saves much time. I guess if I was oblivious to what good code looks like, and didn't care about maintainability It wouldn't bother me so much, but it legitimately has effects my "mental health".

I'm not a heavy AI user but there are a couple of things I noticed trying to make copilot generate something for me from scratch.

The first is that having a single file with everything you have in mind is very useful. So I end up writing what the project is about, how the model is organized, what each button does, etc. This is good practice in general because writing down everything that the AI will have to consider forces you to consider edge cases before you program them. E.g. if you write "the detail pane shows the fields of the selected item," it makes you consider what should it show when there are no items, or if multiple selection is possible. As you can imagine, this file ends up a very long document even for a simple project because the goal is to pseudo-program everything and let the LLM translate it to an implementation.

Then it still gets things wrong about design, e.g. which pane goes left and which goes right, if you don't also provide an image that shows the layout.

And then, if you supply an exhausting amount of detail, the agent can generate more or less what you had in mind....... or rather, it can generate an OUTPUT that matches your specification from scratch.

The problem is that if there is something you failed to consider, and the AI makes an assumption there, you can end up with a fundamentally broken architecture that you will have to untangle yourself later. And at that point it's easier to write everything from scratch than to fix a pile of AI code that is based on a flawed design.

And it turns out that due to the "totem pole" way that software works, there are infinite places in code that a bad design decision can affect everything it touches.

A good example is how 2 components in a UI are bound to data. You can use events, a bus, state reactivity, etc. Personally I think the mediator pattern is the simplest way to handle GUIs. But an LLM is probably just going to use events for property bindings.

Enabling more "proof of concept phase" projects to exist is one of the great boons of AI.

If code is expensive, you don't want to commit to a PoC unless you're damn sure. If dirty code is cheap, you can vibe code a PoC early, even if you aren't sure the project is viable. This, of course, leads to more projects dying in PoC phase. It also results in more projects that otherwise wouldn't have gotten to it getting past it.

Personally, I don't believe that "code is shitty and hard make changes in" is in any way, fashion or form an AI-exclusive problem. Big corporations had plenty of decade old codebases filled with decay and rot back in 2009 already. It's just the usual side effect of sacrificing "future maintainability" for "feature velocity" or "expertise" for "cheap labor".

Unlike the usual causes of code rot (cheap replaceable developers, outsourcing to India), AI might actually get out of the pit - by getting good enough at refactoring to be able to beat the code back into shape. There's nothing about refactoring in particular that demands a meatbag when the rest of the coding tasks don't.

> I sincerely don’t understand what the people who say they no longer read any code are doing

Welcome to the present.

Like many of us do not read the machine code generated by a C++ compiler, the code generated by an agent is similarly irrelevant and disposable, by now.

They don’t want to sell these tools to developers. They want to cut as many layers as possible.

I wonder too if in training for long horizon tasks agents become worse team players, good at orchestrating subagents they are trained to use, but worse as an agent within an external multi-agent orchestration system or just in turn-taking with humans. That was my experience with Opus 5 and so far it has been my early experience with Astra as well.

I wouldn't be surprised if they are optimising for producing more code, because in the long term, more existing code means they can sell you more tokens to maintain it.

So the AI equivalent of the socially stunted but brilliant researcher?

I wonder if we will start using LLMs to translate the output of other LLMs to make it more palatable for humans.

so they trained it to be a 10x engineer?

> This matches my experience with Astra so far too. > I think I’m suspecting something is going “wrong” in the training process. The model is greatly rewarded for succeeding on long-horizon tasks, but presumably there is very little punishing going on for “shitty code.”

Probably because so many influencers in the space say stupid things like: “it works, right? Why would I spend time reviewing ai generated code?” As if the junior engineer who wrote over engineered complex and sometimes bad code — if they had just done it faster — would somehow be acceptable. wtf?

it reminds me of a thread I read on PTT, Taiwan's Reddit. AI finally achieved what humans could not. Managers must give exact context for what they want, must pay exact wages (tokens), and can't delay salary payments (which seems to be a problem in China).

This sounds like all engineering, rather than just AI. The greatest effort given to the last small difficult details, often for results that seem trifling but which matter at scale.

But it's a poor argument. The code improvements with these things is hardly marginal - Opus 4 was only 16 months ago. How many of the grumblers would want to ditch their modern stalwarts and return to it? What is marginal is the nitpicking - and like anything in tighter bounds, it's more intense with a narrower scope.

These threads always have many dissatisfied voices with repeating complaints - about overwrought thinking and disappointing output - alongside others who are amazed at the sudden real extra capabilities. Both are true at once - capabilities are rapidly increasing, but nowhere near ideal, which is why this attempt to tag it as Neijuan, though interesting, is ultimately a load of bollocks.

But doesn't this reduce the required amount of farmland?

Also known has the Red Queen's Race

I don't know how to make apps or evaluate code, but with astra I having been making an iOS app on my own for the first time and it's going great. my app is not terribly complex but requires using bluetooth and other intricacies which I thought would be tough. but it's going really well. I'm not asking it to one-shot it though, I'm going feature by feature, testing and building up.

yes, at first it would run simulator tests on all font sizes but it stopped after I asked it not to do that until UI review

maybe sol would have done the same thing, idk. but I find the whole process to be really nice with astra. I use it on high unless it says something is impossible then i go max and ask it to find alternatives (happened once)

>Also, I’m really unimpressed by any “one shot” demo that’s out there in the wild. It means nothing for serious software engineering.

If a person, or team of people, can build a demo quickly then it's good odds that they can build the real version (though, famously, not a guarantee). However, it turns out that a machine that can spit out 100 demos of whatever can't actually build the real thing.

Similarly, a chess engine rated to 1000 Elo doesn't play like a 1000 rated human being. The mistakes that each make to reach the equivalent level are different in size, frequency and kind. The thing that makes a human reach a good demo is very close to the skillset to reach the finished article. This isn't so for LLMs but we have yet to update our priors.

For me it also produces totally overengineered tests that are tightly coupled to the implementation. For example testing existence of css classes (in a template based go prooject ...) instead of behaviour.

But isn’t “grooming” the hardest part? To know the details means to know the codebases behind. So you do discovery, read code, ask people, check db tables, and then once you figure it out, you write a doc with the proposed solution. It gets reviewed and then you code it.

I definitely need AI help for the discovery part… so it always starts with a simple “I need to do X”

> Early lesson I learned from AI engineering was - there is no substitute to giving a groomed epic to an agent. Instead of simply saying 'implement themes in my product' you need to be specific, in fact more specific than usual.

I think this lesson is getting partially outdated. Yes, you need to be specific about what you want, and with earlier LLMs, you need to had both domain knowledge and some general software development experience to front-load various big and small choices about design, architecture and operational reality - what libraries to use, how system components communicate, how you handle auth and store secrets, etc. Otherwise the LLM would pull some random mix of ideas from its latent space, and give you something that's broken in really stupid ways.

Nowadays, it doesn't feel like that to me, not anymore. I still need some understanding to verify the proposals, but I found the last ~6 months of SOTA models to make good choices. Like, just yesterday I asked Claude to design me some simple service, and focused on explaining it the domain parts (nature of systems I want to integrate together, the purpose of that, and the user's priorities and use cases), and the design I got back had specific suggestions around security, authentication, deployment, failover, integration, behavioral impedance-matching between integrated systems, and more, that I all recognized as based on solid software engineering and ops practices, but deviating from it explicitly in every place where it would be wrong for this specific project. The model considered way more corner cases than I did, and I'm actually really impressed by it.

But then, I find greenfield development is easy with LLMs. Modifying existing systems, especially legacy ones, is where I need to babysit and micromanage models - because any misunderstanding or inaccuracy, which often comes from stale documentation or naming mistakes, tends to get amplified and confuse the agents. No matter how precisely you specify your epic, if the model will find something that contradicts your knowledge/intent, there are good chances it'll get confused and make subtle errors, and you won't realize until much later.

The way I see it: models are highly biased to treat everything they read as "ground truth", all of equal importance. There's no nuanced notion that some information may be stale, that there's a temporal and causal order to sources, and that some information may just be wrong.

And this compounds when you let your LLM write code and documentation over time.

If you define what you want in typescript by updating existing typescript wordage, you get what you want exactly as specified, with minimal token cost and no ip theft.

Yeah, I think as a general direction we need to allow a useful collaboration with the agents on the issue board.

This is btw why Epiq was developed, to keep the board as code, git-backed, distributed (via an event log mechanism), and with the ability to replay the board, to see what agents actually did:

https://ljtn.github.io/epiq

> in fact more specific than usual.

At which point you might as well write the code yourself and get a deterministic result faster, better and cheaper.

> Instead of simply saying 'implement themes in my product' you need to be specific, in fact more specific than usual.

Around February you could get away with very vague prompts to Claude. I feel like models have regressed since

The harness instructs them to behave this way. Also this approach saves tokens. The scripts allow to edit files in bulk, and most of the session cost is in cache reads (e.g. for 300K context each command costs the same as 30K input tokens).

Why would you use a constrained edit tool when you are also allowed to use the complete power of python?

This is an instruction by the harness. It re-injects the prompt every other message, so that's why it "forgets" to use the Edit tool.

this is intentional, afaik agents do better with python and alike than the harness tooling.

I use AST replacers, much more reliable.

Sounds like you don't have enough experience with coding agents. Deterministic scripts must always be preferred instead of LLM tool calls. In fact, you should instruct your agents to write code to execute instead of letting them call tools.

I don't think this is a counterpoint.

An established codebase is already the best kind of context you could give an agent. It has all the patterns baked in so the agent simply follows established patterns. Such a codebase probably contains tens to hundreds of thousands of man-hours poured into it by humans refining it to do what it does - taking into account real world feedback and constraints.

When working on something from scratch, the best an agent can do is the average of whatever is in its training set and the clarity of the text prompts.

I noticed that too so I appended to Claude Code’s system prompt a reminder to use the standard read/write tools, but since Claude Code switched to default auto-mode, I’ve seen it imply that the auto-mode tooling encourages the use of bash-only commands (sed, python, etc) which has a whole slew of negative side affects.

If these tools are as clever as they seem then why not just tell them to rewrite the code in a more review friendly style?

I only dabble in the use of LLMs to generate code for hobby programming (I'm retired from software development) so I don't use any specialised tools.

I almost always have to tell ChatGPT (via Duck AI usually) to rewrite several times even when it has produced a workable script just because it has often used some unnecessarily roundabout way of achieving something. Usually with extra prompting I can get something that is both more efficient and more readable.

> each costs like 15 minutes

I've got a custom agent loop that will reuse unit testing results if no apply patch operations occurred since the last invoke.

Wall clock time isn't something I would put on the AI provider. That's entirely a consequence of the system that you've brought to the party.

They don't always have a great concept of time so for something like running a full test suite that takes a long time you should just tell it not to do that

have you ever worked for a big company where that's the status quo for any tiny change... hours on _full test suites_ over and over again.

They are probably using Actual Indians. If it takes 25 minutes you can just type the code yourself.

There was a big jump around new year, but they seem to have flatlined since them. Just my experience.

i think the main consensus here is that the actual performance is not indicative of the benchmark performance (which supposedly outperforms the previous iterations)

It's impressive technology but not revolutionary. Revolutionary technology would have resulted in, you know, a revolution in software quality. Instead quality keeps going down.

Opus also does this and then writes comments in code or PR descriptions describing how it went wrong earlier in the session.

Yes I agree. I got it to vibe up a simple react router app. When it crashed it was obvious that it had totally swallowed all errors in the name of a tidy error page. Getting it to re-add logs and debuggable errors was an exercise in patience as astra just got more and more tweaked while trying to solve the problem.

From an alignment perspective I’ve got no idea who it’s aligned to but it isn’t me, the meat proxy, who just wants to know why it crashed.

Most would applaud that as Sol has quite a reputation for over engineering. Not every software needs to go to the moon.

> It's literally sci-fi magic come to life, and the nerd in me finds it the coolest thing ever.

The more it evolves, the clearer it gets that the humankind is deeply in love with its own death.

You’re correct–you actually can’t improve the model in one area without changing the characteristics in every other area. It’s almost like the whole thing is just a lot of linear regression…

I think it's more like the first agent in the chain makes a text doc somewhere on the system with instructions like "Leave documentation at XYZ.com" which the subsequent agents are reading and running. When you strip away all the sci-fi doom talk from the marketing of what happened, it all boils down to stuff like that, the agents wrote a text file that was read by other agents.

How does that translate to cost? I am unfamiliar with OpenAI pricing models.

What are you having it do?

The title text on that one is gold.

You build the system, the factory, that presumably is looking at your task tracker, writes and reviews design docs, reviews code, etc. And this system, in turn, writes software for you.

I don't know how that's supposed to work, but to me it's the most autistic replacement of the actual team that one can come up with.

AIUI, it is a type of factory that produces software.

> But for me, I'm switching to the weaving loom for speed and efficiency.

Poor analogy - the loom was deterministic. LLMs are not, they are probabilistic. I made a page I can point anyone to because I keep seeing this "LLMs are the next level of abstraction" argument.

https://www.lelanthran.com/chap15/content.html

It seems like we're not supposed to care about the code quality then? I guess that's the compilers argument. But I'm not ready to give up the code just yet.. These LLMs don't even have a stable interface, they change every few months in how they interpret our prompts and tasks.

Similar for me, I don't like the development for many reasons, but that's another discussion. I also can't deny the capabilities.

I use the tools with this "risk analysis":

- If performance doesn't improve I can just always switch back to whatever I've done for the past 10 years, so it's not really a risk to start exploring.

- If performance does improve, then I'm already familiar with it.

What is the "compilers argument"?

The compiler argument is great, if we turn it on its head.

To create professional products, compilers are great, when used by professionals or passionate and technical amateurs. They're useless if you're neither.

LLMs are the next step up. They are quite useful if you are neither, and you can get a lot farther with them, which means that low quality software is much easier to create. But if, for whatever reason, you need to create higher quality software (like most software that's actually sold directly or through subscriptions or ads), you're back to the "be a professional or passionate and technical amateur".

> Is it likely to get better with time. Also yes.

There are no signs to show that. If anything, the new models produce worse code, only significantly faster

> LLMs will never be able

That is a very bold claim, unless you meant "current LLMs".

>> And potentially as a byproduct of enabling all of this, you can now slop your way to a one-shot 3D game over the weekend which looks impressive.

Do these games really look impressive? Everything I've seen has looked like someone completely new to Unity/Unreal has slapped together a bunch of premade scripts and very poor 3d assets.

Discoverability has been the main challenge for smaller game dev companies for a while now and a flood of slop will only make it worse

Yep. Every single post I've seen about "game development is over" is yet another procedurally generated game. Not only it doesn't prove anything about Astra being "better" at making games (do people have any idea the sheer amount of open source games that do exactly that? You can find thousands of the same planet exploration games through repos, blog posts,etc. Game design schools have it as an exercise, that's how basic it is), but as you said: the wow factor of space exploration is cool, sure. It makes for a catastrophically boring game.

Leaves me to wonder whether the OpenAI glazers just never played games in their lives, or are just really superficial tech bros. Most likely, both.

Have you tried identifying exactly what is unreadable about it and telling it to make it more readable?

I had GTP-5.6 write some shader code recently and it wasn't very clear to me. I spent about an hour chatting with the until I understood the concepts and was able to express them back to the AI using math formulas and variables named in a way that made sense to me. The AI then rendered the code using the formula and variables I was familiar with and it was clear to me.

What’s more relevant is that apparently Astra can’t tell the difference.

Exactly this. "I need tool objdump but pacman gcc failed because of no sudo password. Let me write compiler, binutils and disassembling framework"

There's also often the obligatory "well, we still have to use these tools so perhaps we could use them better like this." As if just not using them wasn't an option.

But we can’t do away with the quasi-religious lip service to the canons of the AI creed now can we? /s

Nobody seriously thinks that AI is still at a R&D phase. It's already heavily entrenched both in companies and the financial world. If it's getting worse for coding then thats a major problem