Codex logging bug may write TBs to local SSDs (github.com)
342 points by vantareed 10 hours ago
b--l 9 hours ago
Codex is one of the most infamous examples of slopware. Just having the window unhidden on my mac will cause it to use 100% of the GPU displaying the spinner message.
THE SPINNER MESSAGE CAUSES 100% GPU USAGE ON AN MBP M5!!
So any time you're waiting on the model (which is 90% of the time), your fans will be blasting (careful, don't use it on battery).
The issue is on github and close to 6 months old. Probably since the release of vibe coded junk. I would literally fix it myself but it's closed source for whatever reason.
There are many discussions about which model is better, or if vibe coding is even possible. I point you to the extent of what one of the most well funded, money flush, well staffed model making companies can do with vibe coding.
To me a screwup this bad (where the CEO has already made it clear they're now "focussing on coding") indicates that there's something truly broken in the company. No one on polymarket expects them to have a leading model any time soon for example.
It's a tragedy. The world needs competition to anthropic.
jofzar 8 hours ago
> Codex is one of the most infamous examples of slopware
Woah, let's not forget Claude code is right there
me551ah 4 hours ago
Claude is also weird for being the only coding assistant that for some reason doesn't support AGENTS.md. Codex, Amp, Cursor all of them support it and read from it, but not claude which forces it's users to use CLAUDE.md instead.
The issue is the higest voted issue on their gitlab repo: https://github.com/anthropics/claude-code/issues/6235
ValentineC 4 hours ago
anon373839 4 hours ago
chorkpop 3 hours ago
datsci_est_2015 21 minutes ago
hexsprite 3 hours ago
bandrami 2 hours ago
kokada 7 hours ago
Not that Claude Code is much better, I just hit this issue[1] because it seems setting DO_NOT_TRACK=1 seems enough to get a really strange behavior in the newest versions of CC.
[1]: https://github.com/anthropics/claude-code/issues/69238#issue...
Edit: I think I misunderstood OP, they're saying that CC is even worse and not better than Codex CLI.
varjag 6 hours ago
Right, just yesterday I found my laptop kinda hot. And what do you think, it was good old Claude deciding to load a few cores with completely idling prompts.
mvATM99 8 hours ago
Yeah exactly.
I'm not exactly building TUI's every day, but even i felt pain when i read that "small game engine" post
matheusmoreira 5 hours ago
TacticalCoder 5 hours ago
sambcui 2 hours ago
I don’t know if you can resonate, but I feel like the Vibe Coded codex and Claude Code desktop apps are iterating way faster than they should be.
malfist 2 hours ago
iLoveOncall 5 hours ago
Surprisingly Kiro is fine (I work at Amazon but not at all on the Kiro team). I prefer it to anything else I've tried (except Amazon Q Developer in IntelliJ, but it's now deprecated).
epistasis 3 hours ago
r_lee 7 hours ago
if we are at 10x with AI and near AGI or ASI, then how is it possible that these products (Codex, Claude Code CLI) are still such garbage?
shouldn't this "agentic AI revolution" have long solved this already?
no way they're over there saying "we are on it plz wait" or that "it's too much effort"?
thewebguyd 2 hours ago
> shouldn't this "agentic AI revolution" have long solved this already?
Daily reminder that Anthropic took over a year to fix the Claude Code terminal flickering issue despite proclaiming all over the internet that software development as a "solved problem."
Apple forked over $250 Million in a class action over false advertising for Apple Intelligence. When do we start seeing the same for the misleading and outright false claims coming out of the frontier labs about the model capabilities? At this point the marketing is doing more harm than the technology itself because its warping the perceptions of those at the top that make decisions. The only reason tokenmaxxing was ever a thing was because marketing mislead execs and technology decisions were made based on vibes instead of evidence.
igleria 7 hours ago
This is the biggest elephant in the room I have seen in my decade+ career. At the same time, look how bad Apple is in software compared to its hardware... It's not an AI only problem, it's almost like software in general gets a free pass on being very unsafe or low quality because no one wants to face the same "profit reducing red tape" that civil engineers or similar face.
CharlieDigital 6 hours ago
thewebguyd 2 hours ago
forshaper 3 hours ago
hombre_fatal 5 hours ago
Like anything, you have to decide between polish vs switch to any other task in the queue. If you choose too much from the latter, then polish suffers, yet that's a human thing.
Also, Codex and Claude Code aren't as bad as people say. I think most of the noise is embellished by the "hah see? AI sucks" angle.
It's kind of like how HNers would claim to your face that you can't actually build anything with Javascript and Node.js (JS just sucks too much), then they'd list off a few footguns that were supposed to demonstrate why. In other words, champing at the bit for JS to lead people to catastrophize issues that were pretty mediocre.
geodel 2 hours ago
coldtea 4 hours ago
jeffybefffy519 6 hours ago
Because vibe coding is a toy… thats the secret.
You can use it to accelerate development certainly, but that requires careful change->review cycles. The developer still needs to be in heavy control, versus vibe coding having an agent own the code base.
layer8 43 minutes ago
The issue is that apparently AI coding means that developers stop caring about software quality. Which puts the whole purpose into question.
ValentineC 4 hours ago
The "AI revolution" feels like it's creating a bunch of ultra-smart AI models are scarily good at cracking most of human-created security (Mythos), but also happen to be careless snobs that just leave litter and mess in their wake.
mnicky 5 hours ago
If the code churn is high the investment to refactoring etc is less beneficial than may be obvious. I don't remember the details but I heard in some podcast that the code base of Claude Code changes so fast that any piece of code won't be there for long..
coldtea 4 hours ago
tartoran 4 hours ago
fg137 6 hours ago
You are asking too many good questions.
user43928 6 hours ago
The products generally work just fine on my MacBook.
I have not encountered major issues in either the Claude Code CLI, the Codex Desktop app, or Claude Desktop app.
They generally get the job done. I don't measure disk writes or analyze the GPU usage.
Zababa 6 hours ago
A simple explanation is that they are "good enough" for most people and they have better things to do. Even if tomorrow I was 100 times as productive, I still wouldn't have time to do literally everything and I would have to prioritize.
coldtea 4 hours ago
nicce 7 hours ago
Not only Codex, but I can't leave ChatGPT app in macOS open for few hours, because it will consume 60 gigabytes of RAM over time and crashes all the apps.
Mindboggling. Or can't use Google's AI Studio in browser because it takes 100% CPU.
Need to write own app for everything???
nbaksalyar an hour ago
It's not just Google AI Studio, it's also Google proper. Just one search result page consumes gigabytes of RAM. How did this happen? I've switched to DDG and never looked back.
veber-alex 5 hours ago
ChatGPT works ok for me but Whatsapp consumes 1000% cpu after the mac wakes up after sleep.
I swear a few years ago shit like this didn't happen on macOS.
coldtea 4 hours ago
porridgeraisin 7 hours ago
the damn chat.openai.com webapp lags a lot as well on long chats, typing takes so long.
rsfern 5 hours ago
giancarlostoro 3 hours ago
> It's a tragedy. The world needs competition to anthropic.
I agree, though Sam Altman's company is the last option I'd want to replace Claude with. I would sooner exhaust every open model.
CryZe 4 hours ago
> THE SPINNER MESSAGE CAUSES 100% GPU USAGE ON AN MBP M5!!
This seems to be a common Chromium problem across tons of software. GitHub has the same issue with its spinners, VSCode as well.
xpct 8 hours ago
Well thank you for your service. I thought about trying out Codex after the disaster that is Claude Code. I'll be fine without either one on my machine
jofzar 8 hours ago
Imo codex is significantly better then Claude code for me ATM.
christophilus 5 hours ago
Codex is much better, which is to say, it’s only pretty bad.
comboy 7 hours ago
I mean, Codex CLI is really bad. But Claude's CLI is so much worse.
Welcome to the world of tomorrow!
markdog12 5 hours ago
This software has been terrible for me. Burns tokens like crazy, and fails. Most times I try to use the browser plugin, it just says it can't use the plugin. When it does work, it takes minutes to click a button. Unusable workflow.
I ask to generate a png with an alpha channel. It can't. Instead, it outputs a chroma-keyed image, then generates a python script to remove chroma key (fails), then a js script (which also fails). Then my 5h allotment is up.
It's frustrating because if it worked as they advertise, it'd be an amazing tool.
EMM_386 4 hours ago
Although they can technically do it, I wouldn't be asking LLMs to generate binary files like PNG with alpha channels, no matter how simple that may seem. If it's easy enough to manually create one yourself, I would do that.
The best way for LLMs to do this is likely to write a scratch program (which is what it seems to have reached for in the second half), write code (which they are good at) and have the library create the image.
At some point it is just easier to handle such things yourself, and use them with text-based formats.
fps-hero 2 hours ago
> THE SPINNER MESSAGE CAUSES 100% GPU USAGE ON AN MBP M5!!
One conspiratorial idea I had was that this isn't a bug, and that Codex was actually doing computation on users' hardware under the guise of "thinking". Like Folding@home, or bitcoin mining malware, involuntarily on paying customers. Your usage is being subsidized by your personal compute hardware that you can't take advantage of unless it was being applied at massive scale.
This would make even more sense when you consider that thinking and response time metrics aren't publicly being tracked. There is an assumption that LLM interaction is being processed as fast as possible, but this doesn't align with the reality of fixed hardware and oversubscription. Of course throttling is occurring. So, if you can take advantage of local compute, delay the responses and you have even more access compute!
I find it difficult to believe that given the scale, number of users, and money involved, that someone hasn't fixed this "bug".
CSMastermind 2 hours ago
Lol this was my theory as well.
l33tman 9 hours ago
This was fixed long ago, if I'm thinking of the same bug. It was stuck in an inf loop all the time the codex window was open.
cncjvu7 8 hours ago
Nah it's still doing weird shit. Uninstalled that crapware last week.
seviu 7 hours ago
To be fair with Codex, you can use any harness you want with it. Access is not gatekeeper by a crappy full of slop electron app.
So just move to PI, or whatever.
Claude on the contrary, forces all plan users to use their horrible app, which, if you ever dared to use cowork, only once, will run a 2GB VM on app start, no f's given. at all.
Not justifying it. But if you use the official Codex app, thats on you. If you use the official Claude app, it's because you are forced to.
Sidenote unrelated to the post: since the Fable thing, and after serious thinking, I moved to open source models. I still have the basic OpenAI sub, but then easy lifting is now done elsewhere.
coldtea 4 hours ago
>if you ever dared to use cowork, only once, will run a 2GB VM on app start, no f's given. at all.
Of all the issues, this seems like the most tame. I mean, there are single Chrome tabs that can use 300MB or even 700MB. A 2GB VM for what is likely isolated local testing of scripts and commands or local lightweight first-level inference to help guide the main harness sounds reasonable.
thewebguyd an hour ago
Not being able to use my own harness on the subscription plan is my biggest gripe with Anthropic/Claude. For what I work on, I still get better results with Opus than I do with GPT5.5-codex, but damn do I hate that I either have to PAYG or I'm stuck using Claude Code.
drdexebtjl 3 hours ago
I haven’t ever tried Cowork, and Claude Desktop shipped a 10 GB VM image on the tiny internal storage of my Macbook.
No way to remove it without hacks like creating an empty, read-only file in its place.
Having this slop installed and automatically updating is a liability.
xenator 6 hours ago
I have exactly the same problem with Time Machine spinner on macOS. It even doesn't rotate.
Somewhere should be rare specialists with diploma who are capable of fixing such problems with waiting lists for years ahead.
ljlolel 3 hours ago
Building an open source native swift version that doesn’t have that bug: https://github.com/Lore-Hex/Quillcode
tengada1 4 hours ago
I had the exact same frustration and switched to Pi and have had zero complaints
hokkos 7 hours ago
is it closed source ? i can see the rust code in repo contrary to the JS in claude code repo, are you mixing them up ?
nicce 7 hours ago
Codex CLI is the main Rust code. There is Codex Desktop separately, using Electron and the same Codex CLI.
NamlchakKhandro 4 hours ago
Pi mono is the only true harness. Everything else is crap
Supermancho 3 hours ago
If Pi can't use my MCPs, it's too big a step backward. Is the common tooling: https://github.com/nicobailon/pi-mcp-adapter ?
epistasis 2 hours ago
jorl17 5 hours ago
Claude code (desktop) and Codex (desktop) are both absolutely dogshit pieces of software. I can't pick which one is worse. I'd be sort of ashamed to say I actively worked on them, regardless of how they can empower people. Cursor's new UI is similarly terrible. They're all slowly getting better, but too slow for my taste.
They are incredibly slow in unpredictable ways, eat up memory at an insane rate, and just feel like they were built with no regards for UX. Like they crammed together all the engineers with no idea of how to build a coherent and predictable UI and let them loose on the product without proper designers.
The other day Codex (desktop) was eating up 70GB of RAM on my machine. What had I done? Literally nothing. I opened it and let it update once.
Another one with Codex was when I had a specific conversation where no activity was happening and which would make the app spin up all of my CPU cores, rendering it barely usable. It would take seconds to react to anything or update the UI. The conversation wasn't even in focus!!! Restarting the app wouldn't help. After I archived it, it suddenly got better
Claude Code Desktop used to be so, soo, soo slow and eat up so much RAM. It was unusable for anything other than playing around when I first tried it. It also didn't communicate any of what it would do. Using it was like living in a world with no affordances, constantly afraid of interacting with them and being faced with some sort of destructive action. Still, it has definitely been improving in terms of the UI experience.
Cursor's new agents mode suffers from similar issues. Obscenely slow, hogging CPU without anything going on, breaking with existing UX patterns (some of them already well implemented in their other, more polished, previous version), confusing buttons and labels which don't explain what to do and that sometimes do destructive operations on your code.
My favorite cursor absurdity is that if you use their workflow to create a worktree and the worktree setup script fails, the following happens:
1. The agent has no idea that it failed, let alone have any logs of the failure
2. Often you yourself don't get access to the logs of what failed in that script. Don't ask me, half the time it just says it failed with no further logs.
3. When you do get the logs, you cannot copy them in ANY way. You can't even select them. I have had to resort to taking a screenshot to do OCR on it
I've also had cursor repeatedly have concurrency/race condition bugs when creating multiple worktrees in parallel. I have 5 tasks, I spin them up all together so they can create 5 worktrees and they crash with random internal cursor errors. Wasn't the point of this abhorrent new UI you've stuffed me with to enable parallelism?
It's like people aren't even testing the shit they ship. Which I guess they aren't.
I'm a big believer in AI and think it is changing the world and will continue to do so, but I almost get offended at how bad these products for which I am paying (sometimes quite a lot!) are. There's "move fast and break stuff" and then there's "build crap to call stuff".
ljlolel 3 hours ago
That’s why I’m building an open source native Swift version: https://github.com/Lore-Hex/Quillcode
iknowstuff 3 hours ago
I’ve been using Codex and Claude in Zed via ACP. Some bugs but overall very pleasant experience vs anything Cursor.
energy123 6 hours ago
Let me guess, there's also a bug where they train on all our data?
varjag 6 hours ago
They don't need to. You pay them for the privilege to do black box reinforcement learning already.
woadwarrior01 9 hours ago
Someone posted a temporary workaround for this on X[1].
sqlite3 ~/.codex/logs_2.sqlite "CREATE TRIGGER IF NOT EXISTS block_log_inserts BEFORE INSERT ON logs BEGIN SELECT RAISE(IGNORE); END;"
Also, I found that running VACUUM FULL on the sqlite file on my laptop shrunk it from 27GB to a mere 73MB[2].
sgarland 4 hours ago
DB-level rules saving the day once again.
NamlchakKhandro 4 hours ago
The real solution is to stop using it and switch to Pi
woadwarrior01 4 hours ago
I’ve been using oh-my-pi with GLM-5.2 xhigh as the main model and GPT-5.5 medium as its advisor model. IMO, the combo works better than either of those models alone.
christophilus 5 hours ago
Well, everyone's bashing on OpenAI as well they should, but just a reminder, unlike Claude Code, Codex is officially available to customize here: https://github.com/openai/codex
It's fairly easy to patch.
redox99 2 hours ago
That's the CLI, not the codex app which is proprietary.
milkshakes 21 minutes ago
the issue is in the cli and app-server
neuralkoi 8 hours ago
Vibe coding takes "move fast and break things" to a whole nother level.
cryo32 8 hours ago
Yeah. Here I am sitting on a major incident at our company because someone’s vibe coded shit went seriously wrong.
al_borland 5 hours ago
I hope that ends up in the RCA, to show these tools as a real risk, and not swept under the rug, where all blame is shifted elsewhere.
cryo32 4 hours ago
Imustaskforhelp 8 hours ago
Can you talk more in detail if possible and are allowed to do so?
I do know one instance of someone literally losing a job because they vibe-coded their way to prod. Their response/justification was: "The code wasn't written by me. It was written by Claude/Chatgpt"
They hadn't done anything to the database itself but you betcha that there are some horror stories involving database, lack of proper backups and Vibe-coding gone insanely wrong.
cryo32 7 hours ago
ValentineC 3 hours ago
flir 5 hours ago
smoe 5 hours ago
latexr 5 hours ago
comboy 7 hours ago
We are running out of things to break.
stavros 6 hours ago
Make more things to break.
GL26 6 hours ago
as long as you don't have technical debt, vibe coding is mostly useful for prototyping. For a real product, true SWE will never be replaced
Otek 6 hours ago
Already got replaced at world top tier tech jobs. „True SWE” will be niche / luxury soon, just like real woodworking vs IKEA
inigyou 6 hours ago
throwatdem12311 5 hours ago
all code is technical debt
i2km 8 hours ago
Shocking. Been open a week and AFAICT just silence from OpenAI. I just find it baffling. You'd think that these vendors would be very sensitive to this sort of issue. I mean, surely they have multiple agents hooked up to github monitoring potential issues and proposing fixes, right? ...right?
Surely it should be trivial for them to have their own tools spinning away trying to fix all the github issues in real time...
drakythe 2 hours ago
They're pretty bad about fixing issues it seems. My favorite is #2472 which they demonstrated "fixing" on stage on the release of GPT 5, but the ticket is still open and the "fix" hasn't been merged. The original blog that flagged this fact https://blog.tymscar.com/posts/openaiunmergeddemo/ and the issue: https://github.com/openai/openai-python/issues/2472
cl3misch an hour ago
There have been Issues on Github about the same problem since April. I'm using Codex a lot and I'm very happy with its performance (UX and output), but it's baffling they haven't fixed this problem.
taspeotis 6 hours ago
OpenAI really snatched defeat from the jaws of victory late last year when Claude Code was a laggy mess.
Nowadays Codex has typing latency out of the gate, whereas Claude Code has the odd pause but generally displays my key presses as … you know … I press them.
kasey_junk 5 hours ago
Fwiw I have the exact opposite experience.
christophilus 5 hours ago
I find Claude Code nearly unusable. I always have to type in neovim if I’m typing anything more than a few words.
aquariusDue 4 hours ago
It runs fine for me on an old ThinkPad X220 loaded with 8 GB, an i5 and a barely working SATA SSD. This is on Fedora and Claude Code is installed from Anthropic's dnf repo (the latest channel). Granted I'm on the Pro Plan and I'm not running lots of sub agents but the default terminal app from KDE (Konsole) renders and keeps Claude Code responsive enough.
I must be honestly missing some key piece of workflow otherwise I don't know why it would run so slow for other people on better hardware? Granted I'm taking care to tell Claude to not exhaust CPU cores and make sure to not trigger OOM errors, akin to "make no mistakes pls".
jofzar 8 hours ago
This is actually such a classic blunder (shipping trace/debug logging on for everything), but funnily the impact is not in a normal way.
It's crazy we have hit a point where memory, CPU speed and disk speed isn't getting clapped because a Dev shipped logging at trace level instead of what used to the application being catastrophically slow so its immediately fixed in the next update.
kuekacang 7 hours ago
It helps too that agent work is done server side so you can hog all the local resources for your thin client.
bravetraveler 6 hours ago
Somebody please donate some tokens to this plucky startup, they need our help.
joelthelion an hour ago
A good moment to switch to an open solution like opencode or pi.
ramon156 9 hours ago
Blegh, I puke every time I see obviously AI generated comments in GH PR's. You cannot assume any of these people have done their research, other than telling Codex to do it for them
b--l 9 hours ago
It's because they use gpt-5.5-xhigh (the money making* model) to build it.
(*for them)
purpleidea 7 hours ago
I want to like codex, but the quality is just not very good, especially when compared to Claude.
It used to work okay, but a while back they landed a major regression for an entire team of folks I work with.
No response, no workaround.
newtwilly 7 minutes ago
Decent sandbox + sandbox override experience with pi coding agent... pi-sandbox uses the same sandbox tech that claude code uses, although it uses a fork that's a little behind, and I'm not sure exactly why it uses a fork.
You can install pi, then install pi-sandbox locked to the current version. Here it is described how pi-sandbox plus an additional extension allow you to have the experience where a sandbox is used, but you can fall back to unsandboxed with approval required. https://github.com/carderne/pi-sandbox/issues/50
christophilus 5 hours ago
I don’t trust any agent to respect any boundaries. They might today. But tomorrow’s vibe coded slip update might break it in subtle ways.
My solution to this is to only run agents in a sandbox of my own making (a locked down Podman container).
drakythe 2 hours ago
They can't respect boundaries as long as those boundaries exist only in the LLM instruction set. A human being who follows rules long enough the rules will become second nature (usually), almost to the point where long running companies are known for having rules no one understands (Chesterton's Fence is alive and well).
But an LLM have a limited "memory" and while the instructions might land in there and be of sufficient priority to be "respected" a single instance of that memory getting too full or the LLM autocompleting the work around because that was the statistical "best" solution and any barriers that exist only in LLM instructions and not in hardcoded guards will evaporate like so much morning fog.
matheusmoreira 4 hours ago
I went the full virtual machine route. Just finished hardening the setup and firewalling it off my local network. Not perfect but it does make me feel much safer.
altcognito 4 hours ago
I think part of the question should be, why is there no QA or test that catches this? It's one thing to be slopware, but why didn't anything run a test that catches this?
java-man 6 minutes ago
what QA?
theowaway213456 4 hours ago
Every time you write a test that handles some data, you write an assertion about how much data is handled?
Come on, this is such an easy thing to forget to test. Don't act like there is some magical testing strategy that would have caught this
altcognito 4 hours ago
I'll acknowledge that this is probably not likely to get caught.
Integration testing could/should catch this, especially for a client side app.
A simple constraints is a good thing. "Our app shouldn't use more than 50mb of ram, or use 3gb of disk space."
sigbottle 6 hours ago
I have noticed absurd lag from the browser usage and sometimes complete bricking of my network too on my computer. I thought it was just my computer getting old, but possibly it's ChatGPT.
ares623 9 hours ago
i hope they find the smoking gun, the key insight, the kicker.
59nadir 9 hours ago
Then they can apply a clean solve, the cleanest solution.
It's fascinating how offensive some of this verbiage becomes to you when you see it attached to LLM output too much.
jofzar 8 hours ago
Ugh this one's gets me so bad, same with "wire" and "wired" everything is wired to something.
never_inline 29 minutes ago
wrxd 5 hours ago
At least they could call someone who’s is absolutely right so that the tool can see its mistakes now
xfgong 6 hours ago
Same issue with Claude Code btw — it writes massive debug logs to ~/.claude/logs. Had to symlink it to a tmpfs to stop wearing out my SSD.
eddyfromtheblok 5 hours ago
I don't see this. According to their docs, logs are no longer written: https://code.claude.com/docs/en/claude-directory
bob1029 9 hours ago
I'm struggling with how this much logging information could be generated at any level of verbosity. Is codex writing log entries while it's sitting idle? Why would someone want to look at these logs?
dundercoder 9 hours ago
If something like this is helpful or necessary, that’s what ram backed tmpfs is for.
mrweasel 9 hours ago
Using a RAM backed tmpfs would be a work-around as to not trash your SSD. It's doesn't fix underlying problem. It's incredibly poor design on OpenAIs part.
taosu_la 8 hours ago
Can someone tell me if the current sub-agent of codex is available now? There used to always be a spinning issue.
indiv0 9 hours ago
This thread will become a typical "haha slop company made slop" but I've been bitten by a bug exactly like this before in a (pre-AI, artisan) OSS project. The maintainer there didn't properly account for DST when calculating last backup time, so the app started and never stopped writing/re-writing backups continuously.
Perhaps the framing shouldn't be "haha slop" but rather why doesn't the AI write better quality software than we do? To which the answer is obvious IMO -- even emergent properties can't elevate AI intelligence too far above the training dataset. So how do we get to superintelligent (or at least "not-wreck-your-NVMe-endurance-telligent") AI, if we, as a whole, are not smart enough ourselves?
Judge not the slop-bot, lest ye be judged yourself, engineer.
Zenul_Abidin a minute ago
I've been bitten by this bug for several days, to the point where I had had to write a script to delete the WAL so that my server would stop getting locked up from a lack of disk space from codex logging.
You can find it here: https://github.com/openai/codex/issues/28224#issuecomment-47...
I have been making noise about this bug for a week, so I'm glad to see this is blowing up on HN.
sleples 9 hours ago
We've gone from "you're holding it wrong" to "the training data was bad because humans suck too". Difference is, humans learn from their mistakes.
klibertp 4 hours ago
A singular human does (or tends to). Humans as a group, where members join and leave a group with time, also do learn, but at a much slower pace - over the years to decades timeframe. "X things programmers should know about Y" is a template for quite a few very influential blog posts, yet for most of them, you find many programmers, even decades later, who don't actually know what they "should".
My experience was always that 90% of code is ugly and clunky. I'm not at all surprised, while reviewing AI-generated code, to see many of the same ugliness we regularly commit. The quality of the output code is now consistently average, which means it's basically shit in 90% of cases, but it tends to mostly work (in the general case). The same kind of shit I've seen people push to production thousands of times in my career.
We don't fully know how to write good code. We don't really understand what good code should objectively look like. Spending more time on code doesn't automatically lead to better code (but costs a lot more). Above all, we don't need good code - the business side is perfectly fine with "good enough right now" rather than "maybe a lot better half a year from now". And that's what the models are trained on. They would, indeed, need quite a lot of "emergent properties" to go from that to consistently good code. ASI-level properties, I suspect.
SilverSlash 8 hours ago
> Difference is, humans learn from their mistakes.
Great! So next time the human will prompt the agent to watch out for and avoid this bug.
sdesol 4 hours ago
ponector 8 hours ago
xpct 8 hours ago
Lack of accountability is the cause here. People don't think before hitting the 'Publish' button. Their managers let them off the hook because the culture still allows making egregious mistakes, as long as there's an LLM to blame.
applfanboysbgon 9 hours ago
1. I bet that developer only made that mistake one time in their life. Humans learn from their mistakes, LLMs don't. If you rely on LLMs to generate all of your code, you can expect to run into the same issues again and again.
2. "One developer somewhere in the world made a bad mistake one time, so this represents the quality of all software devs everywhere". Maybe they were just a bad developer? Bad developers exist. I have never written a bug that has destroyed my users' hardware, and I think that writing such a bug is completely inexcusable in an enterprise environment with software that will be shipped to millions of users, as Codex is.
matharmin 8 hours ago
LLMs do learn from mistakes. Not as directly from individual mistakes like humans do, but in aggregate the models have improved much more in the last year than most humans I know learn in the same time.
xpct 8 hours ago
Y-bar 6 hours ago
lifthrasiir 9 hours ago
> I have never written a bug that has destroyed my users' hardware, ...
Probably whoever (human or agent) originally decided to put TRACE logs into SQLite also thought---or reasoned---so. Maybe the decision was right at that time but the amount of TRACE logs have increased enormously. You will never know.
applfanboysbgon 8 hours ago
da_grift_shift 8 hours ago
What are your thoughts on the SNR of the linked GitHub issue threads? Consider the volume of comments posted and the substance of each comment.
fn-mote 8 hours ago
I read the first page and they were excellent. Each was clearly written by an experienced dev who knows how to substantiate their claims and propose an acceptable fix that could just be merged.
Your comment, on the other hand, would be improved by including your own opinion on the matter.
gruez 5 hours ago
rvz 9 hours ago
The first of many bugs that are beyond the complexity of its authors, thanks to comprehension debt.
Even with tests, the more complex the code base is, the more risky it is to vibe-code on it without introducing more bugs [0] and increasing the debt. Does not matter if the CI is green or if all the tests pass.
It gets even worse if you can't explain the change / pull request or what the implications are after applying that "suggested" fix.
[0] https://sketch.dev/blog/our-first-outage-from-llm-written-co...
HPsquared 9 hours ago
There are going to be sooooo many consulting opportunities after this wave.
hun3 8 hours ago
The operating system has historically trusted the applications not to do dumb things too much.
Only now we're witnessing the consequences much more frequently thanks to accelerated slop.
skydhash 5 hours ago
> The operating system has historically trusted the applications not to do dumb things too much.
The OS is a thin layer providing an abstract and consistent interface regardless of the hardware configuration. Policing applications is mostly related to security and resources utilization, not moronic errors.
hun3 3 hours ago
> The OS is a thin layer providing an abstract and consistent interface regardless of the hardware configuration.
This is called a hardware abstraction layer, not OS.
abihordun 7 hours ago
SQLite + unbounded TRACE logs = firehose in a bathtub. No rotation, no cap, no surprise. The RAISE(IGNORE) fix patches a design flaw. OpenAI's silence is worse than the bug.
consp 10 hours ago
Why didn't the review process spot this obvious error? Oh wait ... @codex review this
cedws 8 hours ago
Moreover why isn't the bug fixed already? I thought programmers were obsolete now. Surely one of the leading AI labs has figured out full automation of software development end-to-end by now if that's so.
charcircuit 9 hours ago
Because it's not an error. The software is working as the creators intended. The diagnostic data (trace logs) are intentionally being saved for debug purposes.
whalesalad 4 hours ago
Yikes. I have a habit of leaving sessions open for a long time. I just ran `sudo iotop` to watch live disk activity and sure enough all my idle codex sessions were spinning away writing god knows what constantly to disk.
Imustaskforhelp 10 hours ago
I don't understand how Codex can blunder so badly. I imagine that even if they would be using vibe-coding, surely they must have some good engineers. So why is there such severe bugs?
One can argue that these products are the flagship products of their respective AI companies aside from the AI models themselves of course.
I imagine that this story will be picked up by the news left and right, some stories just feel this way and this one is like that (given 12 upvotes on HN in 7 minutes)
The only logical conclusion (from this incident) that I can have is: An (vibe-coded?) product is hard to maintain even for some of the best engineers and is bound to have severe bugs.
2. Proper testing and taking issues seriously is the key if you still wish to do this and there isn't much. This is a week old issue which I can only classify as severe.
I wish to keep an nuanced opinion about it but oh this is bad for openAI (not as bad as them accepting autonomous AI within drones and mass surveillance though)
My point is: AI has both uphills and downward valleys and cliffs. It might as well just accelerate you, which could be, towards your downfall as well. Its recommended to keep an eye while driving and not drive too fast.
AI companies might be like car companies which don't offer a brake pedal.
espdev 2 hours ago
> I don't understand how Codex can blunder so badly. I imagine that even if they would be using vibe-coding, surely they must have some good engineers. So why is there such severe bugs?
I'd say this is also partly a problem of working under intense pressure and the demand to work faster and faster - even faster now with "AI". All these companies are competing with each other very aggressively and are driving their employees like horses in order to win the "AI" race.
dathinab 9 hours ago
> I don't understand how Codex can blunder so badly.
because they trust the AI too much (and seem to be fin with acting knowingly negligent)
the problem is
- AI tends to produces very convincing looking code, even if fully wrong
- AI does mistakes of kinds no human would do, at least no human who is also able to write convincing looking code
- code reviews are hard, a lot of devs, including senior devs, put a lot of implicit trust into the co-worker behaving "sane and non malicious". But AIs behave sometimes not so sane and in a way (wrt. trying to be convincing). In the worst case in ways which if it where a human you might consider to be them trying malicious sabotage the product
Like a "dump" example from work:
- AI randomly removes a HTML element id while doing other changes in jsx/react
- the PR has a lot of changes, the id removal line looks innocent, like some on the fly cleanup
- human reviewers have the bad tendency to often not look too much at deleted lines, only if they need reference to how a new line was before (but it's only a deleted line and no new line)
- you don't expect humans to randomly without reason delete important properties of components when changing other things
- you maybe would still have found it, but it's a emergency fix for a production issue
- it happens to miss integration tests, but happens to still matter a lot for one specific important for complicated reasons not properly tested flow (similar people tend to not test logging too much, at best the presence of needed info but hardly ever the absence of noise)
bakugo 3 hours ago
"Vibe coding" implies minimal to no human involvement. It doesn't matter how good of an engineer the person who typed the prompt was, they were not involved in writing or reviewing the code, so the end result will not reflect their skill. The whole point of vibe coding is making software engineers irrelevant.
People like to go on about how "good engineers review their AI code" but that's just not what's happening in reality. Not only is reviewing large amounts of AI generated code unpleasant and mentally taxing, it also negates most of the perceived productivity boost, so people are simply not doing it.
> Proper testing
There is no formal testing that would be expected to catch an issue like this. It can barely be classified as a bug, the logging is working as intended, just with negative side effects that weren't accounted for.
The only real way to proactively prevent an issue like this is for a human programmer to stop and think about this code as they're writing it and go "hmm, we're logging large amounts of data to disk at a fast pace here, this may be a bad idea". Without human involvement, this is just going to keep happening. All vibe coded software is bloated and unstable, I have yet to see a single counter-example.
PunchyHamster 9 hours ago
> I don't understand how Codex can blunder so badly. I imagine that even if they would be using vibe-coding, surely they must have some good engineers. So why is there such severe bugs?
Because it was deemed not Hard Enough task for real engineer to look at, so AI was sent to do it with no supervision, just checking the effects.
Also overly excessive logging is probably useful to them in chasing some of the edge cases, the cost to users doesn't matter in the slightest to them
supriyo-biswas 9 hours ago
The truth of the matter is that any time that has been saved in writing the code must be spent on ensuring proper system design, reviewing the code, and most importantly of all, QA, which is an uncomfortable discussion for AI techbros who are peddling complete automation of the software profession.