Hacker News

Via the incident page:

> Environment variables marked as "sensitive" in Vercel are stored in a manner that prevents them from being read, and we currently do not have evidence that those values were accessed. However, if any of your environment variables contain secrets (API keys, tokens, database credentials, signing keys) that were not marked as sensitive, those values should be treated as potentially exposed and rotated as a priority.

https://vercel.com/kb/bulletin/vercel-april-2026-security-in... as of 4:22p ET

Seriously. Why am I reading about this here and not via an email? I've been a paying customer for over a year now. My online news aggregator informs me before the actual company itself does?

Last year Vercel bungled the security response to a vulnerability in Next's middleware. This is nothing new.

https://news.ycombinator.com/item?id=43448723

https://xcancel.com/javasquip/status/1903480443158298994

Security is hard and there are only three vendors I trust: AWS, Google and IBM ( yes IBM ). Anything else is just asking for trouble.

> The only reason to dramatically overpay for the hosting resources they provide is because you expect them to expertly manage security and stability.

This and because it's so convenient to click some buttons and have your application running. I've stopped being lazy, though. Moved everything from Render to linode. I was paying render $50+/month. Now I'm paying $3-5.

I would never use one of those hosting providers again.

Yeah, given there insane pricing I think the expectations can be higher. Although I know it is impossible to provide 100% secure system, but if something like that happens, then the communication should at least be better. Don’t wait until you have talked to the lawyers... inform your customers first, ideally without this cooperate BS speak, most vercel customers are probably developers, so they understand that incidents like this can happen, just be transparent about it

Completely agreed. At minimum they should be advising secret rotation.

The only possibility for that not being a reasonable starting point is if they think the malicious actors still have access and will just exfiltrate rotated secrets as well. Otherwise this is deflection in an attempt to salvage credibility.

Welcome to the show.

While a different kind of incident (in hindsight), the other week Webflow had a serious operational incident.

Sites across the globe going down (no clue if all or just a part of them). They posted plenty of messages, I think for about 12 hours, but mostly with the same content/message: "working on fixing this with an upstream provider" (paraphrased). No meaningful info about what was the actual problem or impact.

Only the next day did somebody write about what happened. Essentially a database running out of storage space. How that became a single point of failure, to at least plenty of customers: no clue. Sounds like bad architecture to me though. But what personally rubbed me the wrong way most of all, was the insistence on their "dashboard" having indicated anything wrong with their database deployment, as it allegedly had misrepresented the used/allocated storage. I don't who this upstream service provider of Webflow is, but I know plenty about server maintenance.

Either that upstream provider didn't provide a crucial metric (on-disk storage use) on their "dashboard", or Webflow was throwing this provider under the bus for what may have been their own ignorant/incompetent database server management. I guess it all depends to which extend this database was a managed service or something Webflow had more direct control over. Either way, with any clue about the provider or service missing from their post-mortem, customers can only guess as to who was to blame for the outage.

I have a feeling that we probably aren't the only customer they lost over this. Which in our case would probably not have happened, if they had communicated things in a different way. For context: I personally would never need nor recommend something like Webflow, but I do understand why it might be the right fit for people in a different position. That is, as long as it doesn't break down like it did. I still can't quite wrap my head around that apparent single point of failure for a company the size of Webflow though.

/anecdote

I wonder which tool that is

Huh, curiously; I'm on Arch Linux, crash happens in Google Chrome (147.0.7727.101) for me too, but not in Firefox (149.0.2) nor even in Chromium (147.0.7727.101).

I find it fun we're all reading a story how Vercel likely is compromised somehow, and managed to reproduce a crash on their webpage, so now we all give it a try. Surely could never backfire :)

Sadly I coudn't make Chrome crash here. Would be fun.

Chrome Version 147.0.7727.101 (Official Build) (64-bit). Windows 11 Pro.

Video: https://imgur.com/a/pq6P4si

I use uBlock Origin Lite. Maybe it blocks some crash causing script? edit: still no crash when I disabled UBO.

Reminds me of circa 2021 Chromium bug where opening the dropdown menu on GitHub would crash the entire system on Linux. At some point, it got fixed.

Same with Chrome on Windows 11. I opened the vercel home page using the url once after which it stopped crashing when clicking on the logo.

MBP - M4 Max - Chrome 146.0.7680.178.

No crash.

Now I don't want to click that "Finish update" button.

I'm running 147.0.7727.57 and this doesn't happen. Macbook Air M5. VERY interesting.

Same hard crash on Chrome Windows 11

Do you have a chrome://crashes/ entry ?

> Ones NOT marked as sensitive should be rolled out of precaution

if it's not marked as sensitive (because it is not sensitive) there is no reason to roll them. if you must roll a insensitive env var it should've been sensitive in the first place, no?

Who is this “theo” person and why are multiple people quoting him? He seems to have little to say that’s substantive at this point.

Conjecture, but the wording "limited subset" rarely turns out to be good news. Usually a provider will say "less than 1% of our users" or some specific number when they can to ease concerns. My guess is they don't have the visibility or they don't like the number.

I feel for the team; security incidents suck. I know they are working hard, I hope they start to communicate more openly and transparently.

I remember working support and being told "always say 'subset' unless you absolutely know it's exactly 100% of customers" lol

The lack of details itself is telling enough. Whatever comes out will be no doubt PR sanitised and some bigger clumps of truth won't make it through the PR process.

He doesn't work at Vercel but he is the type to never pass up any opportunity to chase clout.

I don't know if I'd trust some random programmer-streamer-influencer on anything other than the topic of streamer-influencing.

Based on what, "feels like it"? Claiming that Cloudflare is affected by the same hack has to come from somewhere, but where is that coming from?

Ah, Theo with his vast insights and connections into everything. That man gets around, and his content is worth it's cost.

Theo's content boils down to the same boring formula. 1. Whatever buzzword headline is trending at the time 2. Immediate sponsored ad that is supposed to make you sympathize with Theo cause he "vets" his sponsors. 3. The man makes you listen to a "that totally happened" story that he somehow always involved himself personally. 4. Man serves you up an ad for his t3.chat and how it's the greatest thing in the world and how he should be paid more for his infinite wisdom. 5. A rag on Claude or OpenAI (whichever is leading at the time) 6. 5-10 minutes of paraphrasing an article without critical thought or analysis on the video topic.

I used to enjoy his content when he was still in his Ping era, but it's clear hes drunken the YT marketer kool-aid. I've moved on, his content gets recommend now and again, but I can't entertain his non-sense anymore.

”Any host” of what? That’s such a non-descriptive statement and clearly not true at face value.

I do remember that OpenAI did use Vercel a year ago. They might have likely moved off of it to something better.

> @theo: "I have reason to believe this is credible. If you are using Vercel, it’s a good idea to roll your secrets and env vars."

> @ErdalToprak: "And use your own vps or k3s cluster there’s no reason in 2026 to delegate your infra to a middle man except if you’re at AWS level needs"

> @theo: "This is still a stupid take"

lol, okay. Thanks for the insight, Theo, whoever you are.

Is AWS security boundary the AWS account? Are you expecting Vercel to provision and manage an AWS account per user? That doesn’t make any sense man, though makes sense if you’re a former AWS employee.

You forgot the source to backup your claim

> Next.js is the new PHP, but worse, since unlike PHP you don't really know what's server side and what's client side anymore. It's all just commingled and handled magically.

Wasn't unheard of back in the day, that you leaked things via PHP templates, like serializing and adding the whole user object including private details in a Twig template or whatever, it just happened the other way around kind of. This was before "fat frontend, thin backend" was the prevalent architecture, many built their "frontends" from templates with just sprinkles of JavaScript back then.

People say "Next.js is the new PHP" because it's the most popular and prominent tooling out there, and so by sheer number of available targets it's the one that comes up the most when things go wrong like this.

But there are more people trying to secure this framework and the underlying tools than there would be on some obscure framework or something the average company built themselves.

Also "pay a real provider", what does that mean? Are you again implying that the average company should be responsible for _more_ of their own security in their hosting stack, not less?

Most companies have _zero_ security engineers.. Using a vertically-integrated hosting company like Vercel (or other similar companies, perhaps with different tech stacks - this opinion has nothing to do with Next or Node) is very likely their best and most secure option based on what they are able to invest in that area.

Next.js is the polar opposite of PHP, in a way.

PHP was so simple and easy to understand that anyone with a text editor and some cheap shared hosting could pick it up, but also low level enough that almost nothing was magically done for you. The result was many inexperienced developers making really basic mistakes while implementing essential features that we now take for granted.

Frameworks like Next.js take the complete opposite approach, they are insanely complex but hide that complexity behind layers and layers of magic, actively discouraging developers from looking behind the curtain, and the result is that even experienced developers end up shooting themselves in the foot by using the magical incantations wrong.

I recently got hit by a car on my bike. While I was starting the claim filing process the web portal for ICBC (British Columbia insurance) was acting a little funky / stalling / and then gave me a weird access error. Down at the bottom of the error page was a little grey underlined link that said “vercel”.

I’m not exactly surprised, but it seems like the unserious, ill-informed and lazy are taking over. There is absolutely zero reason why a large, essential public service should be overspending and running on an unnecessary managed service like vercel… yet, here we are.

Your entire recent posting history is "software engineering is over, AI has won."

What's your agenda here?

LOL. Attackers will run these agents but the thousands of maintainers will be so dumb to sit idly and get hammered with exploits. I wonder what the ratio of attackers to maintainers must be, 1:1000 is a fair assessment i take it.

Also LLMs will be used to attack only, no one will be smart to integrate it into CI flows, because everyone is that dumb. No security tools will pop up.

Slop coding and makeshift sites being thrown up with abandon at breakneck speeds is going to buy me a lot of minivans.

>> ai is going to lead to mass security breaches.

Let that be the end of Microsoft. Was forced to use their shitty products for years, by corporate inertia and their free Teams and Azure licenses, first-dose-is-free, curse.

ShinyHunters are a phishing group. What does this have to do with AI agents?