Hacker News

Countless directors and managers are now cosplaying as engineers. I've seen so many myself and that's just my tiny slice of this engineering world

To be fair, "people with no skills inflating their own value" is what LinkedIn has always been like. But I guess LLMs are uniquely well positioned for that task.

> People that previously have barely any experience in tech, now being hired in AI startups because they are good bullshitters.

I like the symmetry of people being unable to detect "skill" in people who are great at manipulating language and in LLMs who are also great at manipulating language.

I heard the term "Sloperator" a few times. Quite fitting...

This isn't new due to AI, just amplified.

For a long time I was anti-licensure in tech; now with the bar being lowered to next to nothing, it seems as if licensure is more important than ever — not to protect this trade (though it will do that, and that is a benefit), but because the sheer amount of irresponsibility in the usage of LLMs and “AI” in general begs for licensure and adoption of a regulatory body for software in general.

Yes, I've found that reminding yourself of how they actually work helps keep you on guard against LLM-patterned mistakes. Especially things like carefully considering what parts of the current task likely fall outside the distribution of corpus + RL data (as much as that can be guessed).

> ...Unfortunately people sometimes get defensive against this take. But I think treating the LLM as you described can make you a better LLM user and help get better output. It helps understand the failure modes better, and moderate one’s reliance on them. Just like how we should do for every tool we work with.

B...b...but the Anthropic trainer said we'd get the best results if we don't think of it as a tool, but instead give it a name and think of it as our brilliant coworker!

Why should I trust you, internet rando over a stormtrooper-level salesman? /s

> Brandolini's principle

I much prefer the alternative name: the Bullshit Asymmetry Principle.

The best way to describe the LLM intelligence is "an expert system that works the way people thought expert systems would work". You can encode a massive amount of skills into an LLM, and then the LLM uses those to navigate problems.

But the LLM is still dumb where those skills doesn't have good coverage, since unlike the expert systems it maps fuzzily to its skills, and they are tuned to produce results over rejecting the request when its unclear if coverage is good. As long as that is true you have to treat them as dumb even if they sometimes produce brilliant results.

Turing-completeness is a necessary pre-requisite for being able to fulfill the requirements of a Turing machine, nothing more. In the same way that cell division is a necessary condition for life, but cell division does not mean a given life form itself is sentient.

Intelligent life-forms can generate probabilistic outputs based on inputs, but being able to generate probabilistic outputs based on inputs is not what makes us intelligent.

The rate of progress can be high and they can also be dumb next token predictors. Not sure why that is hard to understand.

These models can do a lot of things but they also can't do a lot of things. In order to use these models effectively you have to understand that they are next token predictors and how that allows it to do what they do.

Not sure what your point is? Those things can both be true.

Or should the discourse in a diverse community like HN only reflect the positions you personally hold?

Not a dichotomy actually. Highly depends on the task.

Would be nice to get high karma commenter votes count only ..

Opinions differ. This is not news.

It's almost as if there were many people using this site, and there is no clear consensus on LLMs, so people from various camps upvote interesting stores to support their cause. And people who are still somewhat undecided upvote both, if they present good evidence.

I mean even perennially contentious topics will get this behavior.... some thing about emacs makes the front page, within a day or two there will be a vim post up there. Same with Rust is (good|bad), or if systemd creates an even more awesome tool, the haters will come along and recycle stories about bugs from over a decade ago.

There's a lot of people here. Not all of them read it every hour, and discussions like this among large groups often take a very long time with lots of repetition. Human group dynamics (aka politics) is slow.

> So tired of this discourse and this site.

You're welcome to leave if you don't like it. The site was like this long before you joined, and will like it long after you leave I'm sure.

It's also worth noting, that an awful lot of math discoveries are perfectly in line with dumb next token generators - they are finding a way to formally construct an argument and being surprised when it doesn't work, or surprised at the outcome of the grind. Not all of them are made by brilliant leaps of intuition.

"Dumb next token predictor" keeps popping up since that is the core way they work. Since they aren't logic engines but prediction engines they will always return a result regardless what you ask it.

Some predictions might be the tokens "I don't know", but that is based on the model mapping your text to those tokens by having seen many similar "I don't know" responses to such contexts, it didn't do any introspective logic to produce that "I don't know", and its possible it actually does know if it followed another branch there so "I don't know" is often not even true.

If they had an introspective part that stops the prediction when its too unreliable it would no longer just be token prediction engine, and I believe we need such a part for them to become what I call smart. I don't think LLM will ever stop being dumb without such an introspective part to them.

And no, that introspective part is not a part of the token predictor. At least not in us humans, the feeling of certainty we have is not a prediction, it is bundled with our thoughts, so we get both "answer is a bear" and "certainty is low", we don't get just one of those as a "prediction".

Will LLM become smart as humans with such an introspective part? I don't know, but I think they will never become as smart as humans without one.

Note: The certainty score has to be per conclusion or response, not per token. You can't evaluate a responses validity by aggregating the weight of each token. Meaning its a logic engine, not token engine, that evaluates the certainty of a statement being correct or not instead of a token being correct or not. That is the level human thinking works at and seems to be dramatically more efficient.

> Now back to AI for software creation. "Review is the bottleneck because EVERYTHING must be judged by a human." No. It doesn't. We just need to build AI review systems, that will do reviews better than (or at least as good as) humans. The human review quality bar is far below 100%. Far far far. If we can show (likely in the next 12-24 months I think) that AI review quality is consistently above the human review quality - again, based on evidence, based on real data - then that's it, then there's no good reason to have humans review the code.

Let me know once the majority of software engineering organizations start only checking in markdown files and let code be generated non-deterministically from these specs in CI. If this is not happening now, there’s clearly a sufficiently high level of distrust in blind LLM output (both code itself and reviews).

And even then, are you suggesting humans shouldn’t at least review the markdown specs? Why not have LLMs review the specs then? Is there, perhaps, some fundamental quality to human review process that is desirable here?

This reduction of everything to stochasticity is silly. Or, to put it differently: Do you accept a value with some error appearing in your bank account on salary day?

We have plenty of systems where complete accuracy is the only acceptable thing. Computers are great for such things. Until we all get caught up in a way of delusion and start writing those systems as natural prose passed through an improperly understood stochastic machine.

Yes but the key here is doing proper risk assessment. "What is the consequence if the LLM gets this wrong?" "How do we verify the output?" "What are the legal ramifications for using the LLM in this way?" "Who is responsible when the LLM fails?" "Whats the expected accuracy here?" etc.

In the current AI mania, there's a lot of due diligence simply being ignored. Plenty of "Well humans make mistakes too!" going on here on HN too.

This is what I truly don't like about the CVE system.

One such example is CVE-2023-45853 [1]. Zlib included in it's source an extra set of utilities and add-ons. One such utility, MiniZip, had a buffer overflow vulnerability. BAM, 8.8 CVE (was a 9, looks like they pulled it back a bit). But not one that the 99% of applications using zlib would ever be vulnerable to because almost nobody used the MiniZip utility. It was so unused that the solution for zlib was to simply remove it.

I know about this one particularly because our security policy required us to do a BUNCH of pointless updates for it since zlib is in just about everything.

[1] https://app.opencve.io/cve/CVE-2023-45853

Yes, the CVE system is broken. LLM-generated CVEs is making it even worse.

I suppose all these fake issues and the many more that have absurdly elevated severities could be considered an attack on the system itself, stripping it of credibility.

Yes. This was the case long before LLMs sprayed the tire fire with gasoline. See e.g. this 2021 gem from Dan Abramov (React's creator) https://overreacted.io/npm-audit-broken-by-design/

CVE-2026-4I5109 - Your kitchen drawer contains forks, which can maim and/or kill. An attacker with kitchen access could access the drawer. See...

CVE-1972-404: The code is written in C. ^_^

The majority these days seems to be real issues, but probably not exploitable. The latest LLMs are very good at reading code and finding issues where the code is wrong. They are somewhat less good at finding exploits - both because finding exploits is harder than finding code that is clearly wrong (even if not exploitable), and also because the better ones are intentionally made to not create exploits (since they don't know if you are a good or bad - they assume bad.

However if I was writing this response just one year ago I would instead be saying: the majority off LLM CVS are noise where the code is correct, and often they are writing up for code that doesn't even exist.

Which is to say I suspect the repo in question was generated with a year-old LLM, since they act like that. The new ones [mostly?] are much better.

Still, if a modern LLM points out something you should fix it. Even if we can't figure out how to exploit it today that doesn't mean we won't figure it out in the future.

> The vast majority of CVEs are not exploitable, basically noise.

This is not true if you consider security-in-depth. Many of them are exploitable on their own but maybe not in combination with other issues that are as yet unknown or known but not patched everywhere.

As a simple example a local privilege escalation issue that is not exploitable on a device that only I ever have access to, essentially becomes a remote root access flaw if you have untrusted or unreliable users (clients with accounts for instance) on the system. This works on a finer grain too, seemingly minor issues spread through the kernel and user space can add up to a serious exploit.

I once had a customer threaten to reject a delivery over a CVE. I dug into it. The CVE was that an Ocaml PostgreSQL client didn’t implement an authentication feature.

I convinced the customer to accept the delivery by pointing out that (1) our app had zero lines of ocaml and (2) the feature had been implemented in the ocaml driver since the CVE was issued.

>The vast majority of CVEs are not exploitable

Perhaps in isolation. The issue is when you can chain exploits to bypass multiple layers.

Not exploitable, or not exploitable in isolation?

Are you trying to say LLM’s are tuned to only discover unexploitable bugs?

They’re not bad if you give up on being able to reproduce arcane bugs. Don’t pin versions, pull everything from latest, CVEs are just a rebuild. Certainly not an amazing idea, but it solves the problem of the day.

The funnier, Kafakaesque problem of the day is interactions with mandatory cooldown periods on new versions because of supply chain risks.

I’ve had a couple tickets get stuck because the CVE scanner says I have to update, but the cooldown enforcer says the version hasn’t been out long enough.

In our organisation, my security team spend the majority of our effort weeding out unexploitable CVEs - which are the majority of them. But the volume is growing year on year - I don't know how sustainable it's going to be.

Snyk routinely tells us that we need to upgrade urgently because they found a vulnerability in the Windows version of a package. We run everything in Linux.

Eh, when we see this from Microsoft, Linux kernel, and Apple all at the same time just thinking it's everybody being dumb becomes risky.

Now, if you're an F100 that's currently not using Anthropic it's pretty likely you can go to them now and ask for a non-guardrailed unlimited Mythos trial for a month you can get an agent that will not only find bugs for you, but also write exploits in a framework that will demonstrate them in action. Once you see an AI chain a bunch of shit together, get a foothold and then expand around laterally in your system you start getting concerned about how you're going to keep up.

The time has come to pay for our sins of shit security.

We can shoot for 100% accountability by demanding 100% accuracy.

Of course what will really happen is anyone looking for problems will stop going to the source for fixes and will start gaining cred in the dark markets of exploits and problems, which is better for everyone, right?

Just because someone is using AI doesn't mean they didn't find a bug, it's more likely they just have no means of determining if it's a real bug or not, so to the maintainers tickets seem random. The problem comes in when clever attackers can take these untrustworthy tickets and run them through another LLM that determines if an exploit is possible and then uses the attacks it finds before anyone has a chance to patch.

Currently we either get AI promo vulnerability dumps like from Chrome with pretty graphs that no one checks or false positives.

Where is this one now that was hyped everywhere?

https://news.ycombinator.com/item?id=49133889

The GitHub submitter could no longer reproduce the issue and the LKML post has no replies:

https://lore.kernel.org/all/CALCETrXbj__SFQMzPZhES5y6-sh4np-...

What are the requirements to become a CNA?

What do you mean? It isn't a DoS to assign every single bug fix a CVE! Every single bug is making someone vulnerable in some way.

And this year, some of the reports suddenly got good and meaningful. https://daniel.haxx.se/blog/2026/04/22/high-quality-chaos/

So the agents started doing something useful after a period of filling mailing lists and bug bounties with slop. Sound good, but that's not entirely a good thing. The volume of good reports is a burden as well, and it's likely that long-lasting open source C/C++ projects have legitimate vulnerabilities unpatched. But we don't have any new maintainers, I think.

It's going to be very very difficult to build this system as the exploitability of any particular CVE can massively vary depending on your system configuration.

There are a lot of things that are bugs, but cannot be exploited in a standard configuration, most people would wonder why this is even a CVE.

But then you have those users that would have the application, then something like a report module that was imported, and another 3rd party module that imports images, and maybe another one that arranges the images in a chart. And suddenly that isolated CVE is now exploitable by user written data because of the original bug, it just required a complex and unexpected chain to get there.

> I'm very curious what organisations would have such a policy.

I would humbly suggest any org of any size that has insurance cover that covers anything tech related (e.g. data loss/recovery, cyber etc.) has a very good look at the small print.

Over the last few years insurers have aggressively been adding "no vulnerability patch, no claim" exclusion clauses.

Many orgs (esp w ISO27000) have a vulnerability management policy that involves patching at least critical CVEs within a short timeline. Tools like trivvy make it possible to do the scans…

I have been given a list by security. "We had an automated tool scan that machine. It reported these. Fix anything medium severity and above.

Never mind that some of them involved vulnerabilities in some part of the bluetooth stack (servers in our datacenter don't even have bluetooth). But they just didn't care

SOC2 CC7.1 [1] requires a vulnerability scanner, findings tracked with tickets, assigned severities according to a documented risk-based system, severity-based SLAs for remediation, and that the SLAs mostly be complied with or have tracked exceptions.

However it doesn't mandate any particular SLA, or the details of how risks are to be evaluated.

Organisations get to write their own policy, and they don't need to commit to patching every CVE within 24 hours or anything like that.

[1] https://www.compliancebase.org/controls/soc-2/cc7-1

If I remember correctly, we had to patch or provide justification for CVEs flagged by tools like AWS Inspector for SOC2 as well.

Any org large enough to have separated the people responsible for the security exposure of the organisation from the developers with familiarity of what's deployed is likely to have done exactly this.

The thing you have to remember is that CVEs can be a) scanned for without exerting mental effort, and b) counted.

It's quite common in enterprisey environments.

For one thing, bigcorps in regulated areas like it a lot. They push hard to get it required by the regulations (in practice if not directly). Although it's quite inefficient, it becomes a regulatory moat. A cost they can bear that potential upstart competitors cannot.

It's more common than many think https://old.reddit.com/r/kubernetes/comments/1vb3x2c/where_a...

Many large organizations like banks have requirements like this and they solve it through a mix of automatic scanners, e.g. Trivvy, and self-deception as not all systems are actually scanned in any sufficiently large org.

> I'm very curious what organisations would have such a policy

Do you provide SOC2, HIPAA, GDPR, or similar certifications to your b2b customers? Then your tech stack undergoes an annual audit, and in your audit you will need to provide a paper trail for every single vulnerability in your stack.

In practice, this means that your audit compliance software (something like Vanta.com) is going to be setup to mandate every CVE in the whole stack is patched within SLA.

ITAR

If by "terrible" you mean "amazing". Imagine being able to tear down an entire Jenga tower of bullshit corpospeak, and replace it with actual security.

Nothing new about that

Yes, but getting an exception approved can be a huge pain in some places.

If you work with the government, you don’t really have this luxury

You'd be surprised...

That is if it didn't completely delete the repo in the first place. Consider yourself lucky

I've created bug reports to open source projects that have related files in my repo. Then I can just list the reproduction instructions like:

1) arrange xxx, such that yyy.. or git clone this repo where this is set up. 2) ..

I thought that might have been quite helpful not just to the person I'm sending the bug report to, but also to myself when I need to evaluate if the bug has been fixed.

OP suggests something at NIST changed in 2024, where they stopped doing as much verification as they did before.

To be sure, the suggestion is not funding cuts, but an increase in workload with same funding.

Here is the 2024 NIST announcement OP links to: https://nvd.nist.gov/general/news/nvd-program-transition-ann...

> Currently, we are prioritizing analysis of the most significant vulnerabilities. In addition, we are working with our agency partners to bring on more support for analyzing vulnerabilities and have reassigned additional NIST staff to this task as well.

The OP described this as "NIST effectively hit pause on deep analysis. "

It does sound like they stopped doing something they used to do in 2024. I personally have definitely not spent a lot of time directly interfaced with NIST on cybersecurity in decades past, I know nothing about it, just what I read in OP.

Are you saying the OP was wrong to call what NIST used to do "deep analysis", and/or that the thing NIST stopped doing was "purely ministerial/clerical" in a way that it would not have caught fake reports anyway, contradicting the OP? Or other?

Again, to be sure, the OP's suggestion was not that this was caused by NIST funding cuts, but by "a massive surge in vulnerability reports,"

Hilariously, this comment tripped the LLM writing filter and was auto-deaded. I vouched for it.

Scout's honor!

The comment is making fun of a Claude-ism where it becomes super “honest” about stuff.

It’s a joke but there is an underlying real effect where this type of language is psychologically manipulative and I would guess makes people believe LLMs output more than if it didn’t use “honest” (or “load bearing” or whatever super serious important sounding word).

You're absolutely right, I have hallucinated this. Would you like to find some real CVEs next?