Hacker News

Meh. A world where the defenders and attackers are both omniscient is a world without exploits. Steps towards that world are steps towards more security. The reason the exploits are being found so rapidly is that they are mining all the bugs left in these projects from decades of coding. Eventually they will run out.

Pretty soon people will just airgap their stuff and that will eliminate most of the attack surface short of a sort of Mission Impossible style operation.

I mean really, given how AI is being marketed, what is the point of the internet going forward when all its contents are going to be ai slop anyhow? Just disconnect and run local models if all you are getting is slop anyhow. The original purpose of the internet is now dead. Real people communicating and sharing information with eachother? Ha! That is no longer valued.

In fact actual innovation might no longer be valued anymore. What is innovation but opening Pandora's box and potentially seeing a disruptive competitor take your slice of the federal reserves money print? Better to nip that in the bud and control all the devils we already know, from a pure sociopathic profitmaking standpoint, which seems to be a very dominant viewpoint among the people in charge of the worlds power structures right now.

> Nicholas Carlini, for example, whose name is on many of the recent high-profile Mythos findings is not just some random dude with a Claude sub on his credit card .... he's an experienced security researcher.

I don’t think Mythos is hype for all kinds of reasons.

Anthropic is a young company but their track record is solid; they don’t seem to hype things just for the sake of hyping things. Sam Altman at OpenAI? We already know his track record…

I’m going Occam’s razor here: the simplest explanation is usually the correct one.

Anthropic had an “oh shit” moment when they realized what Mythos can do. They decided to do the responsible thing: give the industry a heads-up and an opportunity to use the preview to identify and fix the most dangerous zero-day vulnerabilities.

Since the FAANG companies have billions of users, it makes sense to start with them.

There’s still going to major issues for users of systems too old to get patches or updates. Or for IT organizations who think Mythos is a replay of Y2K, where, compared to the warnings, not lot happened.

The bottom line is someone with Mythos won’t need to be an experienced security expert to cause real problems. That’s kind of the point.

Over time that will change. Technology has proven time and time again that as we add a new layer of abstraction over the fundamental functionality, knowing the previous layer quickly becomes vestigial knowledge. It is true not just in software but absolutely all technology there is, going back to the first fire made or atl atl or rock sling.

> it is over-hyped because like any LLM, it requires a suitable human in the loop to keep the LLM on the straight and narrow, and then to weed through the inevitable false-positives and hallucinations.

"Suitable human" is a dry phrase indeed. ^_^

The hype is "gosh look at all the bad things this brilliant almost conscious tool found!"

The reality: an insecure toolchain for an insecure language with an insecure compiler produced a runnable but insecure binary for an insecure OS. We couldn't be arsed to address any of this before, but now we're being billed the full price of our laziness.

I think it's worth to look at the recent XBOW benchmark: https://xbow.com/blog/mythos-offensive-security-xbow-evaluat... they realized that ChatGPT 5.5 works better so the secret is in the architecture (including humans in the loop).

> likely to be sorely disappointed if/when they get their hands on Mythos.

At first they will be delighted. So much money and time saved. When their adversaries get their hands on their system (with or without Mythos), then they'll be sorely disappointed.

how much do you think it is worth in the bug bounty program

They can’t disclose the technical details yet. They did say a detailed write up is coming.

The gamble is that you can cruise on the senior engineer’s diminishing understanding for a few years until models become good enough that you don’t need any humans in the loop and you can fire all those expensive seniors.

>I'm hearing anecdotes from all over about devs pushing LLM-generated code changes into production without retaining any knowledge of what it is they're pushing. The changes compound, their understanding of the codebase diminishes, and so the actions become risker.

No anecdotes needed, it's entirely happening.

But it's also devs, being devs.

is this exciting?

juniors have been writing code forever that is imperfect and not memorized by the people reviewing

isnt the important thing the mechanisms for maintaining the code?

> I'm hearing anecdotes from all over about devs pushing LLM-generated code changes into production without retaining any knowledge of what it is they're pushing. The changes compound, their understanding of the codebase diminishes, and so the actions become riskier.

I don’t think so.

An LLM can produce higher-quality documentation than most humans. If it's not already happening, when a new developer joins a team, they're going to have an LLM produce any documentation a new developer needs, including why certain decisions were made.

It could also summarize years of email threads and code reviews that, let's face it, a new person wouldn’t be able to ingest anyway; it's not like a new developer gets to take a week off to get caught up on everything that happened before they got there. English not their first language? Well, the LLM can present the information in virtually any language required.

As the models continue to improve, they'll spot patterns in the code that a human wouldn’t be able to see.

Most companies in the world do not have “blue teams”. They barely have any kind of security employee.

Not at all. I’m considering that the amount of vulnerable software in the wild is very, very large, with most organizations not managing their systems properly. Imagine all the small to medium size companies that do not have budgets for a dedicated, talented security team. And all the software that will never be patched. We are at the beginning of the exponential

Suppose the following:

1. Any given system has a finite number of findable vulnerabilities.

2. All findable vulnerabilities are fixable (if not in software then with a new hardware revision).

3. Fixing a vulnerability while keeping the same intended functionality introduces on average less than 1 other findable vulnerability.

4. It is possible to cease adding new features to a system and from that point forward only focus on fixing vulnerabilities.

If all 4 are true, then perfect security seems possible, in some sense. I think some vulnerabilities might not be fixable, if you include things like the idea that users can be tricked into revealing their passwords. If you restrict the definition of vulnerability to some narrower meaning that still captures most of what people mean when they say computer vulnerability, then I think those 4 statements are probably true.

Perfect security might be near impossible in practice because vulnerabilities will get more difficult to find and fix over time, but I think we should expect the discovery of vulnerabilities to eventually become arbitrarily slow in a hypothetical system that prioritized security above all else.

I would rather claim that building a theoretically secure system is prohibitively expensive. At the end of the day, Mythos et al. are just better tools for finding vulnerabilities that will eventually be available to both offensive and defensive actors.

If you imagine you had a vulnerability scanner as fast and convenient as a linter, it would be much cheaper to write secure code right away. Probably not perfectly secure, but still secure enough to make sure finding exploits stays expensive.

another "obscurity": I'm not valuable enough to be attacked, compared with the cost. But what if cost has been reduced a lot?

It's probably impossible to achieve security through correctness, but security through compartmentalization can work. See: https://qubes-os.org.

Genuine question as I’m far less technical than the crowd here. Has this not always been the case?

Thank you. I was about to ask.

Quite strange indeed, given that was one of the main points on their security conference a few months ago.

could be a different type of data only attack, which doesnt override the boundaries

IIRC, the GPU is behind a memory controller, so I doubt corrupting GPU memory alone could lead to an LPE. But I suppose it would give you someplace to store stuff if you can make something else read from it.

I should've added /s.