Northflank Breached (status.northflank.com)
430 points by js4ever 18 hours ago
jonathaneunice 14 hours ago
I hope this is a real wake-up call for teams running anything remotely SaaS-shaped. I count at least six substantial, lightning-speed attacks against multi-tenant as-a-service vendors since September. Not this year--just since last month. PingCAP/TiDB, Fireworks AI, Zeabur, OpenCode Console, Railway, now Northflank. All of them modern "we keep your content safe" players with high hopes and probably pretty good protections.
This is a full infosec emergency unfolding here. No matter how grinding it is to motivate a team to invest in protection or resilience (and it is grinding): We gotta get ready. Stat! The AI-accelerated attacker apocalypse has begun. It's boom, boom, boom, thanks for everything, on to the next. They have us sighted and are firing for effect.
So credential rotation, tripwire secrets, DFIR on standby, solid vulnerability and intrusion detection, real listening to that signal, minimized blast radius, spending caps and time limits and tight scoping on credentials, fast incident alerting... We need to get on a real war footing here.
999900000999 13 hours ago
War ?
Last I checked major companies are still outsourcing essential engineering to save a buck.
A bit old, but how was this practice ever acceptable.
https://www.propublica.org/article/defense-department-pentag...
A multi trillion dollar company( Microsoft) too cheap to hire onshore for DoD contracts.
burningChrome 12 hours ago
Worked at a Fortune 200 company. Any govt project you worked on, you had have to have a basic civi security clearance. Any govt contract we got specifically stated you couldn't have off shore folks working on any of the code. Not sure how MS got around those requirements.
sellmesoap 12 hours ago
ZephyrP 9 hours ago
stackskipton 10 hours ago
ozim 11 hours ago
doctorpangloss 4 hours ago
AvocadoPanic 11 hours ago
pwarner 13 hours ago
I wonder what it's like inside the big cloud providers. I think outside Microsoft they've always been security first, but this new era has to be crazy town.
blueblazin 7 hours ago
Google at least has been investing very heavily into cloud security. They acquired Mandiant ($5.4 billion) and Wiz ($32 billion). Many people felt those were very expensive acquisitions at the time, but maybe it seems like a wise move now.
lmeyerov 4 hours ago
awesome_dude 13 hours ago
I mean, Microsoft and Security, those two words have never really been associated with each other in a serious conversation.
They're quite famous for being lax in security - mostly because they're a massive corporation, but also because they're high profile
knorker 13 hours ago
sandworm101 12 hours ago
mlhpdx 12 hours ago
> All of them modern "we keep your content safe" players
But they weren’t and probably aren’t and won’t be. They just put forth an image as much as needed to sell.
ehnto 7 hours ago
It would seem Google account details have been compromised or otherwise correlated, with not a peep from Google, we are getting reports of scam calls to people's real phone numbers about the "business opportunity" available in their attached Youtube properties.
Another possibility is advertising shadow profiles are able to correlate logged in accounts across YouTube and other properties.
theK 12 hours ago
Is this the point where most tech people accept that anything a closed source Software company says about security is effectively a "Trust me bro" as long as it is not backed up by independent verification?
We love to talk rubbish about finance bros but that industry has had independent auditing as part of its DNA for a century[1]
[1]: https://en.wikipedia.org/wiki/Sarbanes%E2%80%93Oxley_Act
lazyasciiart an hour ago
roundup 6 hours ago
Should startups that have experienced a data breach face a lower valuation in subsequent funding rounds?
blauditore 11 hours ago
I wonder which one of these are at least partially due to effortless vibecoding...
Also, if vulnerabilities are so easy to find, it should also be easy to detect them through a simple red team exercise.
0xbadcafebee 9 hours ago
> probably pretty good protections
In my career I've seen maybe two companies have pretty good protections. Both had government contracts so they had no choice. Otherwise there's no reason for companies to secure their stack. They make money easier without it.
The buildings are on fire because they're being built out of straw because it's cheaper. Only a government regulation to build out of brick is going to stop the fires.
SubiculumCode 9 hours ago
I for one have backed up to offline storage our research lab's data in preparation...
Den_VR 13 hours ago
A war footing?
ttul 13 hours ago
Yes. Literally AI is being used by actual nation state adversaries to compromise bits of infrastructure that companies rely on to run their services. They are compromising this infrastructure specifically because they seek to gain advantage against their adversary (us).
krapp 10 hours ago
jen20 13 hours ago
> ...the operational and economic condition in which a nation, military force, or organization mobilizes its resources, industries, and policies as if an actual state of war exists...
Yes?
Den_VR 10 hours ago
therein 7 hours ago
> SaaS-shaped
The usage of shape here triggers me.
roncesvalles 2 hours ago
Your words are comment-shaped.
notpushkin 25 minutes ago
adriand 10 hours ago
Maybe Bill Gates has a point when he says that models capable of significant cyberattacks need to run on controlled infrastructure. Is the era of open weight models you can strip safeguards from turning out to be so great? From my perspective, this kind of sucks.
blurbleblurble 10 hours ago
Sorry but how in the world would that help with anything? You can't control an adversary, that's not how adversaries work. So that makes what you said a literal cop out.
adriand 10 hours ago
tomhp 9 hours ago
Buddy you're nearly 20 years too late for Bill Gates to be making any decisions at MS.
dangoodmanUT 18 hours ago
Unfortunately for infra startups, I think we're going to see a convergence back to hyperscalers as a result of incidents like this and railway.
There's no amount of "nicer to use" that can replace security, at least certainly when selling to enterprises
digitaltrees 16 hours ago
I actually think it’s the opposite. I think people will move back to their own hardware with OSS deployment like dokku. If everything is more bespoke the attack surface is smaller as is the pay off. Hacking railway got access to lots of api keys, hacking some random vps gets less payoff.
roadside_picnic 13 hours ago
> If everything is more bespoke the attack surface is smaller
This is exactly the problem that LLM assisted attacks solve. For a human attacker, the extra work to figure out a bespoke solution is too time consuming to be worth the effort. LLMs can search for vulnerabilities and spit out bespoke attacks like it's nothing. On top of this the bespoke solutions weakness leans heavily into the AI attackers advantage: response time. Even if they can just get your company's single Anthropic key, if they do it at 2am while your one infra person is on vacation, they have plenty of time to extract enough value for their effort.
I agree with the move to hyperscalers. The future (well, actually present) is going to be constantly seeing new attacks happening. The only real solution to this going to be having teams of people and agents constantly watching for these and able to deploy fixes and respond to risks immediately.
digitaltrees 12 hours ago
elevation 12 hours ago
itissid 16 hours ago
How is that ever going to work out? The speed of discovery for LLMs is so fast, these places are getting breached more often than not due to a human errors. Its really hard for machines, let alone humans, to guard against a lot of LLMs going against them 24x7.
vb-8448 16 hours ago
ang_cire 14 hours ago
ijustlovemath 15 hours ago
nine_k 14 hours ago
burnte 14 hours ago
This is absolutely happening. I'm not a developer in SV, I'm an IT leader out on customer land, and I can tell you a LOT of companies are looking to bring things back on prem for security and cost reasons. And before anyone laughs, yes, you ABSOLUTELY can save money going on prem, I've done it for company after company. It's one reason the PE firm that I've worked under for a decade kept moving me to new companies.
digitaltrees 12 hours ago
TedDoesntTalk 14 hours ago
kube-system 13 hours ago
This is misguided thinking. You are less likely to hear about on-prem breaches in the news just because they are less likely to be newsworthy incidents, but they are even more likely to happen. Part of the reason people moved to the cloud is because it is much easier to have a team manage updates and security at scale.
Being a small target is not helpful these days, attacks on on-prem software today simply scans the internet, or uses databases by people who have already scanned it, and is automated.
On-prem software has not been spared from the recent spat of security incidents.
heisenbit 12 hours ago
Imustaskforhelp 16 hours ago
My comment[0] tries to explain the nuance and I would agree with you and both the GP in some sense.
For Entreprise, the shift will be towards hyperscalers as they want trust and less blame (Nobody got fired for picking AWS)
For consumers, we care about pricing and stability and control and we will move back to our own hardware or cheap VPS providers and self-host things.
The middle ground would be where the slight pain could be at in my opinion.
happyopossum 15 hours ago
SoftTalker 16 hours ago
debo_ 13 hours ago
It's probably both.
wat10000 14 hours ago
Cost of hacking a custom setup is going down rapidly. There was broad agreement that the PHP vulnerability was very difficult to exploit in practice. It took two hours for someone to get into Steal My Bitcoin. Apparently all they did was tell a decent model "go exploit that." AI isn't free but it's not that expensive.
TedDoesntTalk 14 hours ago
malux85 14 hours ago
No thats an old way of thinking because it assumes mass targets are better because hacking is a high effort activity. But with the automation of intellect the unpatched systems (which a much more numerous when everyone self hosts) is the key point.
Mass credential stealing is just the top funnel of hacking and (compatativly) isnt that lucrative - these credentials are then sold on the black markets to the next step of hackers who use them to exploit targets.
With unpatched systems and self hosting, this second group can just set an llm at a target and have a much higher chance of success without needing credentials
It would be a dream for the infiltration hackers to have everyone self host. Unpatched and miscondigured systems would be everywhere and LLM automation could make mass scanning them and exploiting them very attractive - not mass scanning and exploiting them for credentials stealing, no! For layer 2 stuff
mac3n 17 hours ago
> There's no amount of "nicer to use" that can replace security, at least certainly when selling to enterprises
Azure would beg to differ.
happyopossum 15 hours ago
None of my customers / clients / coworkers have ever used Azure and "nicer to use" in the same sentence.
InsidiousWombat 15 hours ago
FLeXMurphy 17 hours ago
A moron could build a better UX and DevX.
minraws 17 hours ago
bix6 18 hours ago
Are hyperscalers actually more secure though? I have only heard anecdotal thinking eg nobody ever got fired for choosing IBM.
baxtr 18 hours ago
It’s the same thing: let’s say Microsoft or AWS got hacked. Then you’re definitely not the only one who got hacked.
So, you can always say: Look, it’s not us (especially not my org!). It f’ing Microsoft/AWS! Everyone got hacked.
You and your career are gonna be just fine.
SkyPuncher 15 hours ago
codegeek 17 hours ago
nrmitchi 15 hours ago
InsideOutSanta 17 hours ago
temp0826 14 hours ago
esseph 17 hours ago
Krasnol 14 hours ago
macNchz 17 hours ago
With a lot of PaaS type vendors you are effectively exposed to both the vendor themselves and the public cloud they run their stack on top of.
Broadly I think you're much more likely to be the victim of a platform-level breach involving an immature startup PaaS than a hyperscaler cloud, however I think for small teams the alternate risk you take on is that AWS/GCP/Azure are full of landmines and extremely easy to misconfigure in insecure ways if you don't know what you're doing. Especially if you have AI telling you confidently wrong things.
inglor 18 hours ago
When I was at Microsoft we took security cripplingly seriously to the point we could not deliver features and our competitors could because the security process was convoluted and had multiple reviews and approval steps.
Veserv 17 hours ago
tptacek 17 hours ago
loloquwowndueo 18 hours ago
kube-system 13 hours ago
Unquestionably. They have top tier security operations that are well funded. They follow (or set) security best practices, they are audited, and they fix them quicker than you can. They have inside industry knowledge that is only shared among top organizations in this space, including government cyber ops. This gives them the ability to fix things before they are even publicly known.
You can get close if you have either:
1. the ability to spend hundreds of millions on security operations yourself
or 2. the ability to airgap the systems you are securing
jareklupinski 17 hours ago
> nobody ever got fired
thats the crux; are you working for the paycheck, or for the truth?
could be both, but pretty much all "i was right" stories start with "everyone else said i was wrong"
bix6 16 hours ago
keeda 17 hours ago
Yes and no. They (or anybody else) are not guaranteed to be secure, of course; see all the compromises regularly found in cloud providers; Azure especially has had so many cross-tenant vulnerabilities (there’s one on the front page right now!) that I’m amazed the concept of public multi-tenant clouds still exists.
But they are also very highly incentivized to be more secure. They have millions of customers (hence a correspondingly higher expectation of trust) and billions in revenue from the trillions in economic activity happening on those platforms (hence a correspondingly larger target on their backs.) This means exponentially higher liabilities. And that probably matters more than any other factor.
Imustaskforhelp 16 hours ago
kragen 16 hours ago
The things that Northflank and Railway were apparently breached to steal aren't just things that hyperscalers would have protected worse instead of better; they're AI API keys, things that only existed because people were using hyperscalers instead of running inference on their own hardware (or on Amazon AWS EC2 GPU instances).
tptacek 15 hours ago
People have been breaking in to hosting companies to steal bulk secrets since long before there were inference tokens to steal; the modal attack probably targets database DSNs.
sanderjd 13 hours ago
Why do we think hyperscalars are likely to be immune to this?
kube-system 13 hours ago
They aren't immune, but they almost universally have better security posture than any alternatives.
sanderjd 11 hours ago
asdff 17 hours ago
Theres always a market for seedbox providers. Hyperscalers will always wear the white gloves and enable this market.
wg0 15 hours ago
What about Railway? Some big incident happened?
chi_features 15 hours ago
llama052 13 hours ago
WarmWash 17 hours ago
It's also going to be the downfall of open models.
If you think governments (US and China) are going to tolerate people using models for anti-social behavior, I have a datacenter to sell you.
logicchains 17 hours ago
This will be the downfall of the torrent protocol. If you think governments (US and China) are going to tolerate people downloading copyrighted games, movies and music without paying the rightsholder, I have a datacenter to sell you.
WarmWash 16 hours ago
InsidiousWombat 15 hours ago
azure just had a huge leak
Imustaskforhelp 16 hours ago
IMO, There is no "middle-ground" to the situation.
I wanted to write a blog post about it as I was thinking the same but I guess a comment will help me get my thoughts across just as much.
Either we are gonna see a convergence to hyperscalers as you mention, or we would go back to completely indie companies which compete on perf/price.
There could still be some value to "nicer to use" certainly but with AI, we are gonna see almost every product as being "good-enough" if I am able to convey my sentiment.
An anecdote is: Proton's an middle-ground company. I am seeing the Verge article which states that their CEO has gone a little crazy over the use of AI and the wording makes me lose trust within it. Proton's whole point is security and yet it's having this, It has some stocks in non profit and is a swiss company. It shouldn't have been this way and would've been one of my least anticipated companies but I guess they want growth at all costs and the growth has been slightly lesser than expected and now they are burning the good-will in many ways.
I have to wonder that even if proton might have good engineers, then within this pressure by their CEO/CTO. I cannot trust the engineering talent because of these issues because they are very much forced by their company to use AI.
Proton is also really expensive.
Now just today, also, with a company so big as binance getting hacked and losing 6.5 Billion$. My trust within security just because its a large company is less.
On the other hand, I know a guy called Vikas on lowendtalk who operates his own mail platform called mailbig. I know he vibe-coded the landing page and might've vibe-coded something but it is really cheap. I also know francisco whom I have talked to who runs namecrane. (I think mailbig uses namecrane in many ways)
Perhaps E-mail can be a sensitive matter because you have to place trust in the provider even more because the protocol is not really that much secure but suppose it's an matrix instance (where it is genuinely encrypted) then that shifts it even further. I do trust NameCrane and NameBig from what I can gather.
I'd much rather support them and their cheaper deals than some other middle-corp. Sure some security factor can come in but also, the difference feels comparatively less and the balance has been made
The market will slightly squeeze as Hyperscalers will introduce the solutions of the middle-ground and the new players will everyday enter into the field with more competitive pricing just as well.
So in short, Enterprise will move towards the hyperscalers. Generic Consumers will move towards the cheapest option. The moat of the middle ground where UI/UX was the differentiator is being turned down. There would be some mix and match and there will be cost-savings for Enterprise to move from hyperscalers but it would depend on a lot of factors.
esseph 17 hours ago
I have news for you. A lot of enterprises are moving a lot of workloads back on prem and are expanding their datacenters and on prem GPU compute. Also replacing a lot of SaaS with internal tooling.
justincormack 14 hours ago
If you look at the numbers for revenue at the hyperscalers right now this move doesnt seem significant.
esseph 11 hours ago
esseph 17 hours ago
They are also getting hacked, and they are much jucier targets.
nisegami 17 hours ago
>There's no amount of "nicer to use" that can replace security, at least certainly when selling to enterprises
We can lower our standards instead
dangoodmanUT 13 hours ago
A trick i've started doing is encrypting my env vars, then hard-code the decryption key in the code
that way if the provider gets popped, they need my binary to decrypt it
it shouldn't be necessary and it sounds stupid, but in cases like this, it proves invaluable (it appears they don't have customer code/binary access, but maybe that wasn't important enough to mention)
steelframe 10 hours ago
I think this is a great example of temporarily effective "security through obscurity." It's also a trick that will only continue to work so long as it doesn't get adopted too widely.
roncesvalles 2 hours ago
It buys you time. I had my Anthropic key on Northflank set as an env var as-is and only found out about the incident from the HN front page 6 hours later (I didn't even get an email).
I rushed to the Claude platform console and was expecting to see a 4 figure usage bill or something. Luckily there was no usage.
But imagine if I were traveling. Even a rot13 of the key would've bought me enough time to rotate.
2 lessons:
- don't name your env var "CLAUDE_API_KEY". Call it "ARG_1" or something
- at least do some obscurin' so that the value doesn't start with "sk-ant..."
jamesbfb 10 hours ago
At this point, I kinda feel like the obscurity is our best mechanism for mitigation :(
dudinax 6 hours ago
heresie-dabord 6 hours ago
tty456 5 hours ago
Give it about a week until AI agents add his "tip" to their training...
Ey7NFZ3P0nzAe 11 hours ago
Been doing something like this for a while with a single self contained .sh file that contains a python script:
harvey7 7 hours ago
lock your env vars in hardware. even the whole server and code is compromised, the secrets are still safe.
so you dont need env2 to decrypt env1.
https://seal.zone/ does this
solyin 6 hours ago
That's interesting but if the Azure hypervisor on the server hosting the VM is off limits then it's kind of false sense of security. They protect the key via a sidecar AMD SEV-SNP VM, and if you can't break into the hypervisor, sure, there might be an attack that doesn't involve doing that and only doing lateral movement, but an actual attacker isn't going to respect that boundary.
Geof25 10 hours ago
I will give you one better - encrypt the variables and move the secrets to a different server running hashicorp. Then load the secrets and keep them only in RAM.
pwarner 9 hours ago
Or env1 is the key to decrypt env2
ikrenji 9 hours ago
if the provider gets popped doesn't the binary get popped too? then simply use an LLM to reverse engineer the binary and get the key?
GCUMstlyHarmls 3 hours ago
If its a targeted attack, they'll know and probably "crack" it. If its just data trawling, you might get owned one day if they bother, but likely the env `AWS_KEY=encrypted_abc` looks reasonably close to `AWS_KEY=unencrypted_abc` and any API key probing will just look like it got rotated already.
I think siblings are correct that the more widespread this is, the less effective once every dump is run through a "also look for keys in the binary" step.
zmmmmm 12 hours ago
This could all go multiple ways but one of them is we end up in a place where technical security is just not possible to implement any more. What does that world look like?
Presumably it means much more behavioral security and inherent friction built in to everything we do. Maybe high value transactions will just inherently have to take multiple days to do, just to slow down things enough that human reaction time can equal what LLMs can execute?
On the technical side, I can see zero trust becoming completely saturated through every layer. The passkey you authenticate with isn't opening a gate to a trusted tier within which components trust each other - even within the same software component. Every internal function call at every layer doing hardware signature verification against continuously minted private keys (since we assume exposure, the lifetime can't be longer than milliseconds now).
It's really interesting to think through what a fundamentally robust security environent does look like if you bake in an assumption that technical security is impossible.
m11a 12 hours ago
When we see a breach of AWS or GCP or leading banks, basically any org that has good security, I'll believe technical security is impossible.
Otherwise, I think this is a issue of vendors, even if potentially most vendors fit into the basket of having insufficiently good cyber posture in AI-era.
sheepjumpmoons 9 hours ago
Isn't Linode pretty darn close? IaaS/PaaS provider. Sure it's not one of the Big Three but that is it's role.
anonymous908213 12 hours ago
There is zero reason to suggest security is impossible. This is like musing "what if architecture is technically impossible" after the nth bridge collapse in the 1800s~early 1900s. At some point we take it seriously as a society. Until then, bridges will collapse out of choice to pursue short-term profit above all else.
zmmmmm 10 hours ago
there is hypothetically impossible and practically impossible
if AI tools get good enough that they can exploit ingrained hardware properties such as subtle timing and race conditions and swarms off them become cheap enough to deploy so that they can persistently exploit low probability events we could reach a state where I think it's actually practically impossible.
anonymous908213 10 hours ago
chrisjj 12 hours ago
> There is zero reason to suggest security is impossible.
One reason: There is zero reason to suggest security is possible.
chrisjj 12 hours ago
> Presumably it means much more behavioral security and inherent friction built in to everything we do.
Ironic, after decades of defrictioning.
hmokiguess 18 hours ago
What is Northflank? I am surprised at the amount of new business out there
gorgmah 18 hours ago
I ran some stats on producthunt a few months ago, there are 22 times more daily SaaS launches now than in 2020, the AI impact on the industry is hard to fathom. The blog I wrote (AI-assisted) if you're curious: https://www.mahl.me/blog/product-hunt-is-swamped-now/
highfrequency 14 hours ago
Nice data, and also curious whether the blog post was written by AI as well?
"Product Hunt has also said the quiet part out loud."
quadrature 18 hours ago
Northflank isn't that new, it was founded in 2019. Its a cloud platform, you can get more details by googling.
jacquesm 18 hours ago
New businesses, not new business.
hmokiguess 15 hours ago
oops … sorry!
whalesalad 15 hours ago
I get like 1 email per week (via hn) from someone rolling out a new 'agentic sandbox' environment like this.
Rastonbury 2 hours ago
I noticed these and have now become blind to these
shmageggy 4 hours ago
I wondered the same thing, so I clicked their homepage:
> Run agents and everything they build.
Oh, it’s 100% Claude-slop. How shocking they got hacked! /s
paxy 18 hours ago
You are on a YC startup forum and are surprised tech startups exist?
jmull 18 hours ago
"amount" it says. No matter the context, there could be more than you expect.
hmokiguess 15 hours ago
That they get front page by being breached and me never having heard of them in my own bubble? Yes.
maherbeg 14 hours ago
I think all of these companies need to let you separate secrets from other env vars and not let you ever read your secrets once they're written down.
highspeedbus 15 hours ago
We're observing a worrisome speed up in security breaches. To me it's clear that weaponized AI will cause more harm than some productivity benefits.
It’s the death of homegrown software.
Sol- 14 hours ago
Seems way too early to tell whether it will be good or bad on balance. These breaches are annoying, but not catastrophic, and in the meantime you can secure software.
chrisjj 12 hours ago
> and in the meantime you can secure software.
... in fantasy land.
acedTrex 15 hours ago
Why would this be the death of homegrown software?
highspeedbus 15 hours ago
It will be impossible to any startup or open source software to grow in such a hostile environment. Anything you put on internet will be attacked by automatic tools with the capabilities of a black hat savant. But unlike hackers that do it for clout and money, a billion parallel agents will hack any random small site just because it’s their prompt.
david_shi 2 hours ago
solyin 14 hours ago
acedTrex 14 hours ago
walrus01 7 hours ago
The same AI that can be used to find exploits in things can be used in an internal adversarial way to self-audit your code before you put it into production or expose it to the world. This will have to become the new normal.
baobabKoodaa 5 hours ago
Okay, cool. Now, tell me how I get access to the frontier models that the attackers use?
walrus01 4 hours ago
throwrioawfo 13 hours ago
> The runtime platform for autonomous software
Well, at least they were honest I suppose
wayeq 12 hours ago
"wait, not THAT autonomous software!"
Scorpiion 15 hours ago
I'm curious what people think of BYOC (bring your own cloud) as an option offering to customers or buying it as a customer. Looking at Northflanks website they list it as an offering, I presume those customers are not affected by the breach.
What are your opinion on the concept? Growing trend that should be encouraged or a mistake for most startups? Would you choose one provider over the other because of it assuming price was about the same?
I get that this is often a super expensive "call us" option today, but in theory it doesn't have to be, so for the sake of discussion let's assume it can be delivered at a reasonable cost for the provider at reasonable price for the customer.
If people from Northflanks is reading this, I'm sorry for the incident and I hope you recover as soon as possible.
captainkrtek 15 hours ago
Depends on the nature of the breach, if its a flaw in the software itself, then perhaps exploitable both in the hosted offering and BYOC offering, though perhaps the BYOC aspect gives a bit of "security-through-obscurity" and less discoverable as a target.
Scorpiion 14 hours ago
Good point, there are of course different levels.
I was thinking of a fully private deployed within the customers cloud account connected to the software provider via Private Service Connect or PrivateLink and then either only available inside the customers private network or only via their own firewall. The point being that it's not publicly exposed.
mritchie712 15 hours ago
I was thinking the same thing on security-through-obscurity.
a hacker might know how to hack your service, but they'd have to find it first.
aloknnikhil 13 hours ago
We are increasingly seeing deployments trend in that direction. But trust goes both ways and putting systems in place to prevent abuse from either party is hard and needs to be thought through.
jdprgm 15 hours ago
The Hacker News front page is just becoming a security incidence status page at this point. Everything seems pretty fucked.
chrisco255 14 hours ago
0 days since last zero day
anuptalwalkar 15 hours ago
Unfortunately, this is going to be an everyday story. More and more bugs and breaches will be discovered and this move fast world more and more will be continued to be made.
Security is a perception, but right now it seems like it needs good PR.
lifeisloving an hour ago
Lots of panic around these incidents. This is a healing process. Companies who sacraficed on security for a long time are incurring the consequences, they'll adjust their posture or go out of business. In a year I think platforms will be a lot more hardened. Its not because AI is unstoppable, its just revealing holes and the importance of hiring competent people.
(or things will stay messy because people will throw AI at the problems instead of hiring real security professionals)
The sky isn't falling.
ankit70 18 hours ago
How these Paas (Railway before) are being breached? Poor choices or something else? Scary Nonetheless. I thought AI advancement will make the self hosting painful. If PaaS are no good, then what can be done?
btown 17 hours ago
Modern PaaS is tough because there's a general expectation that users (and agents) are able to see and read back environment variables in their management UI or CLI - which means that you can't segregate credential values from the public web environment. See e.g. the visibility icons on https://northflank.com/docs/v1/application/secure/inject-sec...
And sure, you can encrypt at rest, and perhaps even enforce that only re-encrypted versions are ever seen by the management/API web server. But fundamentally this display requirement means that if your management/API web server is compromised, users' secrets can be obtained by a process of impersonation.
corford 16 hours ago
Stuff made during the move fast and break things era now doesn't hold up very well in the AI era. Or to say another way: startups are very vulnerable now - this new environment is unforgiving towards cutting corners and/or having inexperienced devs
SoftTalker 15 hours ago
I think the ultimate fallout will be that software "engineers" will finally have to become "engineers" in the real sense, i.e. professionally licensed and personally responsible for the designs they approve and the software they build, whether the actual code is produced by LLMs or humans.
As costs and broader economic impact and perhaps even injuries/deaths related to software breaches continue to grow, there will be growing intolerace for vulnerable software, just as there was intolerance for collapsing bridges and buildings.
And I'd also at some point expect something like a "building code" that applies to software that will be exposed to the public.
tortasaur 13 hours ago
seanw444 16 hours ago
Nope. The solution is to move even faster and thus break less things via AI.
bongofizzbuzz 16 hours ago
corford 16 hours ago
this_user 15 hours ago
trouve_search 18 hours ago
Why would ai advancement make self hosting painful? Apart from increased cost of hardware (which you implicitly would pay for in PaaS as well anyways).
If anything, AI makes running linux boxes much easier than 4 years ago; it's now faster and the learning cliff is gone to get the correct CLI commands and digging into system logs.
roncesvalles 10 hours ago
I had this exact thought yesterday. Shouldn't AI obliterate PaaS platforms like this one since it's super easy to build a bespoke CI/CD process now? Since it's bespoke, you can lock it down as much as you want. Attackers don't even know it's there. It's time to go back to `deploy.sh` and a VPS.
whimsicalism 17 hours ago
platforms are better placed to guard against attackers
pluralmonad 17 hours ago
monster_truck 18 hours ago
if you're still doing that manually in 2026 you probably should not be self hosting anything of consequence or commercial liability
dns_snek 17 hours ago
ascorbic 15 hours ago
They're high-value targets, so that's what attackers are pointing the agents at
reducesuffering 16 hours ago
According to HN of the last year, "marketing stunts." Remember when Mythos preview was announced only 6 months ago
vorpalhex 18 hours ago
When you build something small, you can easily see all the doors and properly guard them.
As you build something bigger and bigger, quickly, with several people building, guarding all the doors becomes exceptionally difficult without a kind of aggressive discipline which is contrary to the "go fast" culture.
Cybersecurity in practice until now, for most companies, has been an insurance problem, not a quality problem.
Such companies are quickly discovering that insurance is a post-hoc partial correction and not a proper solution.
This situation will not improve anytime soon for most companies.
The solution does not require any new magical tools.
carimura 18 hours ago
i'm surprised at the growth of all these neoclouds given the current security risk climate...
dash-44 14 hours ago
Correct me if I’m wrong, but Northflank, Railway etc aren’t neoclouds. They’re PaaS providers which is quite different.
neya 17 hours ago
Run agents and everything they build. Secure execution from the first sandbox
Getting hacked while in the business of selling a secure sandbox isn't really a good look. It seems like can't even trust anything these days. The other day OpenCode got hacked and leaked everyone's secrets inside .env to attackers and now this. Sigh.kragen 15 hours ago
There's no point in attacking the security of a company that doesn't try to provide any.
ksajadi 12 hours ago
i wonder if there is a chain reaction point, when one compromised vendor holds secrets to another one and that becomes the next target. Northflank lists Sentry as their customer. So if Northflank secrets are decrypted and Sentry used them for deployment, you have a starting point to attack Sentry next which then opens up a lot more.
mlhpdx 12 hours ago
Is the Network Infrastructure Hiding Protocol work at IEEE going to help thwart AI assisted attacks, if adopted?
itissid 16 hours ago
So if you want to judge as a customer for selection and figure out how likely a new(ish) PaaS company going to get breached, how would you evaluate them?
1. Ask them about what percentage of their own resources goes towards cybersecurity, audits etc. Would they even tell you.
2. Technical stack maturity. If the details are available, because they won't reveal any of it to you.
3. Other signals: Do they give talks on platform security, who is incharge of their cyber security.
happyopossum 15 hours ago
There is an entire practice of cyber risk analysis, and most of the companies I work with do this type of due diligence as part of their vendor management process.
Yes:
* you do ask about things like budget allocation, audits, and internal compliance processes.
* you do ask about the tech stack, how it is maintained, and what the patch and disclosure policies and SLOs are
No:
*you do not rely on what figureheads say publicly; none of that is binding in the way legal agreements are
And if you're a vendor, your deals desk probably has all of these answers packaged up and ready to go; or even published on a website like this https://spyglass.trustshare.com/home (no idea who that is, it's a random company's compliance portal I found while googling my employers compliance portal provider).
[edited for formatting]
SubiculumCode 9 hours ago
If the banks don't hold, nor will the center.
loa_in_ 13 hours ago
The bigger the task, the bigger the surface, the bigger the risk.
bdcravens 8 hours ago
Not a great way for me to hear about Northflank for the first time.
loveandhelp 5 hours ago
south ass riker maneuver
insane_dreamer 15 hours ago
not a good time to be a PaSS or SaSS ...
yieldcrv 13 hours ago
doppler next?
loveandhelp 5 hours ago
How to cut a piece of the left ribs? \s
Not in English majority speaking place at the moment so not surprised "runtime platform" for "autonomous software" is not immediately understood as the somewhat etymological contradiction that it is. Runtime doesn't even have a definition, really. but autonomous implies independent and not ran / hurried like bison off a cliff...
platform and software
sergiotapia 17 hours ago
So sad...
Unfortunately I'm not longer using PaaS tools. Instead I use Azure for work and vps now for all my personal stuff. With AI it's easier than ever to lock a server down tight and mitigate this kind of exposure.
I'm actually pretty pissed at this now. I have a throwaway appstore app that uses northflank for the backend, now I have to go in and rotate revenuecat secrets, apple secrets, ai secrets. For an app with basically 2 users.
Can I even delete the app on apple's app store or whats the process like what a pain in the ass. I'm pretty done tbh.
yellow_lead 16 hours ago
Azure? Really?
postalcoder 17 hours ago
I feel like these breaches started to accelerate after Anthropic wrote a full post about how easy it is to abliterate glm 5.3 to make a frontier-level cyber model.
Now, to be clear, abliteration was no state secret, but that blog post was pure accelerationism.
tptacek 17 hours ago
That was just 30 days ago. This has been happening for months. Part of the "Black October" narrative is just vulnerability research; Calif didn't abliterate 5.3 to find PHP2Shell.
postalcoder 17 hours ago
Yeah IIRC calif research on php2shell preceded all of that but I still stand by the thought that Anthropic has been the most accelerationist AI company despite its lip service to the notion of safety.
tptacek 16 hours ago
_notreallyme_ 17 hours ago
They are using mythos-5.5, which is still better than an abliterated GLM-5.3.
tptacek 17 hours ago
nater5000 16 hours ago
Duh, the idea that those capable of actually utilizing these models for these purposes weren't aware that they could use these models for these purposes up until Anthropic released a blog post about it is silly. Anybody who is even a bit tech literate has been well aware of this inevitability for basically as long as open-weight models have been a thing.
The timing of all of this is much easier to pin to the creation and proliferation of capable of open weight models. Anthropic calling this out is not the cause for it; it's just an obvious correlation.
paxy 16 hours ago
lol no, abliteration communities have existed and abliterated models routinely published long before Anthropic wrote that post last month.
What’s changing now is that models are getting more powerful.
sandeepkd 17 hours ago
You are running a machine with tons of vendor provided agents/scripts to do god knows what on a cloud provider which itself runs tons of agents/scripts to manage that infrastructure and then you expect to be secure.
And no, before the labs there was a degree of fear for everyone to hack into some one else system. With the labs changing the narrative, anyone can just blame the agents for the breach, whoever did it would only get fame and no blame