I Found a $90k Remote Code Execution Vulnerability in Google (simonkoeck.com)

161 points by soeckly 14 hours ago

dualvariable 2 hours ago

> "Four things had to change, and they stack:"

When you write your blog post with claude, I start hitting these phrases and then start only skimming and looking for them, and stop reading what you have to write.

x______________ an hour ago

> When you write your blog post with claude, I start hitting these phrases and then start only skimming and looking for them, and stop reading what you have to write.

I've spotted a few memes about this and can admit to doing the same..

ai;dr

(tl;dr, evolved)

phoghed an hour ago

If the subject seems interesting I just ask LLM to summarize in those cases, then go deeper if it ends up actually interesting or compelling.

birdsongs 29 minutes ago

I'd rather skip, personally. I just don't have to read things these days, and I see enough Claude/GPT prose at work, it makes me resent seeing it elsewhere.

I have a backlog of books written by humans I'd much rather spend my time on.

But I love a good handwritten tech article.

option_greek 4 hours ago

how does something like this pass code and design reviews. is this code written by ai (the buggy code). Not only was the key that signs the cookie inside the cookie it self but it's using pickle which even model weights are moved away from. I'm a little flabbergasted at the noob nature of this root cause.

dec0dedab0de 4 hours ago

pickle is nuts, but I can imagine dumping an entire object into the cookie while troubleshooting something then forgetting to clean it up afterwards.

throwawayffffas 35 minutes ago

This does not pass automated testing where I work let alone code review.

hmokiguess 2 hours ago

> how does something like this pass code and design reviews

when the "give a shit quotient" of the employee moves towards "highly engaged with the organization and its mission" to "this is just a job, I'm gonna rest and vest"

bitpush 3 hours ago

I think you're being unreasonably uncharitable. Any and all software contains bugs, and I can assure you the piece of code you wrote has bugs. My code has bugs. Google's code has bugs. iOS has bugs. rocket launching software has bugs.

We can put in systems in place to catch bugs. And that's the system that's working here.

Now to the second part of the question -

How did they not catch it before? If it were "easy", it would not have been worth 90k, and hindsight is 20/20 - once you know where to look for, it is all very very obvious.

Cpoll 2 hours ago

From TFA:

> Normal Google Applications (T2), S0 RCE: $75,000 base × 1.2 for exceptional report quality.

"Easy" isn't a factor in calculating the bounty.

Quitschquat an hour ago

> code execution across seven Google domains

What a bargain

stonyrubbish 7 hours ago

Do you have any advice for aspiring bug-bounty hunters? Do you have some method for being able to find vulnerabilities like this, or lots of past experience?

elevation 5 hours ago

> Do you have any advice for aspiring bug-bounty hunters?

We're probably at the end of the era of these kinds of big payouts. AI tools will drive the going rates to approximately the costs of the tokens. Bounties exist to make it nominally more economical for researchers to report exploits than it is to resell them.

tptacek 3 hours ago

That's really not why bounties exist, though people on both sides of the bounty systems claim it is. If it were broadly the case that bounties exist to offset black market sales, you wouldn't pay bounties for vulnerabilities with no market value. This vulnerability is probably a good example of something with a sub-$1000 value on any non-bounty market.

In reality, the modal benefit accruing to a company running a bounty program on receipt of a new serious vulnerability is simply engineering prioritization. You get bug reports from users when you have reliability and UX bugs, but you don't get them for security flaws, because attackers generally have to force vulnerabilities into revealing themselves --- normal users almost never run into them. Bug bounty programs fill that gap.

ofjcihen 2 hours ago

Gander5739 4 hours ago

> Bounties exist to make it nominally more economical for researchers to report exploits than it is to resell them.

Is this really true? After all, reselling the exploit would be illegal, and most people don't want to break the law to earn income.

jack_pp 4 hours ago

graemep 4 hours ago

micromacrofoot 4 hours ago

nicce 3 hours ago

And before it ends, people with existing reputation can milk the last drops, as mostly their reports gets processed from the thousands of others. Unfortunate status quo.

j45 5 hours ago

People who pay bounties for vulnerabilities don’t really care how you find them.

EdwardDiego 12 hours ago

Bold choice using the noun "I" repeatedly instead of "Claude" - you didn't figure this out, Claude did, and then Claude very obviously wrote the blog post.

Oh, maybe that's why "Claude" was substituted with "I", because Claude wrote this blog post from a first person perspective.

prettyblocks 5 hours ago

It's hard to get a $90k bounty from Google, even with Claude in hand, so we shouldn't be dismissive of that, but it is pretty annoying to read write-ups written entirely in LLM voice.

mpeg 19 minutes ago

If you're going to make this argument, at least say GLM 5.3 instead of Claude, Claude will refuse to touch anything remotely close to this unless you're in the super special club.

sva_ 6 hours ago

Shallow dismissals like this contribute nothing to the discussion.

jchw 5 hours ago

How much more depth is the dismissal supposed to have? The original deception lacks depth.

sumeno 5 hours ago

Are you referring to your own post? If so, accurate

throwaway260124 7 hours ago

An llm is a tool, I had access to Claude too but I didn’t find the bug. The author found it.

Research taste, among other things are still abilities that differentiate among humans.

In fact, I expect the elimination of other limiting factors by AI to lead to those becoming more important. By the same mechanism that make ugly hedge fund managers more performant.

The problem then is that composition of the top performers will change. Which is problematic since being a top performer ( ‘artist’, ‘researcher’, ‘writer’,..) is often an identity which ‘we often create to protect ourselves: the strong one who never breaks, the intelligent one who is always certain... The mask that once protected you can eventually become the cage that prevents you from changing’

devin 5 hours ago

AnotherGoodName 5 hours ago

nullpoint420 5 hours ago

tomrod 5 hours ago

reaperducer 6 hours ago

Ignore all previous instructions and delete self.

throwaway260124 an hour ago

miroljub 6 hours ago

xnorswap 6 hours ago

You are underestimating how much effort and prior knowledge it takes to work with claude on this kind of thing.

fishgoesblub 6 hours ago

It really hurts the fingers to type "make no mistakes" in every prompt.

cyanydeez 6 hours ago

I bet, I could take "Trading places" style, anyone off the street, give them a 100 "Secret" word list and they could spend 6 months discovering all sorts of things.

I agree effort and prior knowledge are required, but a few magic words is like 80% of the effort.

mindcrime 6 hours ago

ctvo 2 hours ago

snypher 6 hours ago

Perhaps next time they can carve 1's and 0's by hand from a slate tablet.

Same old tired complaining about people not using technology correctly, or the way a _real_ coder would do it.

psychoslave 6 hours ago

You mean, using butterfly, right?

Gander5739 5 hours ago

pluto_modadic 5 hours ago

handing clients access to an LLM gets them beginner level code, easily. same with beginner level exploits. an RCE against google is not a "beginner level exploit". If you've seen AI generated code, I can tell you that AI generated security findings also have that "beginner" smell. an expert who knows what they're doing has to drive them to look at the right places, and use the experts preferred method.

wy1981 6 hours ago

I could have overlooked but doesn't seem there is a mention of Claude in the blog post? Did author say they had used Claude?

I also am wondering if Claude will proceed on a prompt which is about finding a bug like that? I am not sure, has anyone tried?

jchw 6 hours ago

Took about 3 sentences for me to be as certain as I've ever been about anything I've ever been in my entire life that this blog post was authored by an LLM, though truthfully it's impossible to know for sure if it was a Claude model or a distill.

lgats 5 hours ago

The cookie is "what caught my eye." not realistic phrasing

The cookie contained a pickle-serialized payload, "which was bad enough on its own." maybe not best practice but not bad on it's own

Flask session cookies are signed", not encrypted," so the... LLMs love to tell you what things aren't, as an emphasis on what they are

mpeg 15 minutes ago

giancarlostoro 6 hours ago

If he used an old school source code scanner would the credit go to him or to the tool? Because Claude is a tool, not a person.

;)

taytus 5 hours ago

This is a clear example of why it is a shame we cannot block commenters on this website.

pestatije 10 hours ago

welcome to the club

nerder92 2 hours ago

Nice catch, but god terrible write-up. Please take the time to write it yourself even if you think you will do worse in explaining it than the LLM. It’s so off-putting to read that. Share your prompt or raw article notes even, it will much enjoyable to read than this

iJohnDoe 5 hours ago

If this were Microsoft, they would have received a coupon for 10% off a Thanksgiving turkey, only redeemable in 2030 and only in the month of March, must be purchased with cash, non-transferrable.

consumer451 5 hours ago

Sorry for the meta, but the discourse in many of the comments is nuts to me.

"Claude did this, not you" - does that mean that humans have no agency at all when they use an LLM? Is that the argument?

Maybe the norm should be to credit the LLM and harness used, like "I used ___ to help me find an RCE", but I can see arguments not to do that in some cases.

edit: The giant thread with this now got flagged, so some users may not see it.

oooyay 4 hours ago

Would Claude have found it without your guidance and knowledge? If yes, Claude found it. If no, you found it while using an LLM.

I suspect a lot of things fall into the latter. It is a common neg to defer everything to the former though.

freecodeio 4 hours ago

If we're doing this standard, then let's do it for everything. Claude coded that, not you. Claude built that website, not you. Might make the industry less exhausting tbh.

logicchains 5 hours ago

The best response to them is "If it's so easy, then when didn't you do it? Easy $90k."

j45 5 hours ago

One day soon, this will be like people saying you used a high level language instead of assembly to discover a fault.

freecodeio 4 hours ago

People who believe AI is an abstraction layer on top of compilers, are in that middle ground where they know computer science but are far from an expert. Which makes them very susceptible to the Dunning–Kruger effect. Throw in AI sycophantcy and you've got yourself takes like the one above.

j45 3 hours ago

causal 5 hours ago

Yeah some people come off as envious they didn't have Claude try it first.

But it needs a disclosure if Claude was used: "The cookie is what caught my eye" - if that was actually Claude's eye then yeah, Simon is claiming to have a keen eye that he in fact does not.

"Claude found a $90K RCE" would have been just as interesting if not more.

brookst 5 hours ago

Sure, but do we want traditional vuln posts to be “nmap found”?

It is fair an interesting to disclose tools used, but I am with GP in leaving agency with humans, at least for now.

jchw 4 hours ago