Hacker News

https://en.wikipedia.org/wiki/Timeline_of_Amazon_Web_Service...

Azure nerfed the front door of all Azure and O365 services last year.

All it these companies are great at what they and occasionally fuck up.

On the other hand i can’t remember when there was a serious outage on GCP, unlike AWS/Azure who seem to go down catastrophically a couple of times per year.

> Never heard of this from AWS or Azure.

AWS does it more efficiently; it takes down many startups at a time when us-east-1 goes down.

Yep, we also don't touch them for this same reason.

Yep, agree 100%. Such a stupid move on their behalf.

What was the reason GCP took down a startup previously?

Hetzner and OVH also do this all the time.

It's AWS and Azure that are the outliers and tend not to care too much what their customers do with their infrastructure. AWS is perfectly fine with allowing me to run copies of 15 year old vulnerable AMIs copied from AMIs they've long since deprecated and removed. Even for removed features like NAT AMIs.

AWS normally contacts you first.

Nothing surprises me with anything related to support on GCP. While we absolutely do not need them, I have been through no less than 12 different Account Executives over the last 6y and they're all ENTIRELY and COMPLETELY useless.

They all introduce themselves, beg me to setup a meeting w/them and some sort of engineering resource(s), and they come to a meeting with a canned slide deck that is so absurdly unrelated to us that I just laugh, and then the next time I hear from them it's because we have a new AE.

This is my most recent reply (right after Next '26):

> I really appreciate you reaching out; however, we have met with, I dunno at this point, more than a dozen GCP Account reps, execs, technical teams, etc over the years and there's little to no value for us or you, now or in the future. Please do feel free to invest your time on your other clients. We're good; truly.

I love GCP and its services; we have been very pleased with it over the years, but the human side of it? Fucking sucks and I just don't see why they even bother.

huh- I guess there are two HN submissions with meaningful replies...

I said this in the other thread, we got access to our account back, but even with a Account Rep. and a CSM on our account- it still took them a while to figure out what was going on.

I'm sure it could have been worse if we didn't have a rep on our account.

It's Google. They let you use their services, but the moment you don't fit the norm, they suspend you.

This is the conflict at the center of running a hosting company - make it easy to signup and you get a lot of new users but also a lot of abuse.

Implement anti-abuse measures and you will hit some loud false positives (this may be the case with GCP here).

I don't envy anybody running a hosting co - the internet is a really ugly place under the surface.

edit: to add - AWS are really good here. Must be the ~30 years of retail fraud and abuse experience.

Surely this is automated. They wouldn't waste precious dollars on employing humans just to keep other humans happy.

In my experience, DNS changes are a lot faster than they used to be. There’s some website that has a map that tries to resolve your domain with a bunch of name servers around the world that was pretty neat to look at last time I migrated something.

If only there were a quick and easy way to replicate s3 buckets to an independent provider…

… on the Unix command line …

… to a cloud older than AWS…

… if only …

They run a decent amount of their own compute/bare metal server for customer workloads. But likely still had some critical dependencies on GCP.

There would have been efforts to contact them, but it would have been via their contact method, aka the email they set it up with.

Common ways this happens? They are using a credit card to run their business with no backup payment method. Then the company's contact person is on vacation.

Sign up for terms. It will get you payment terms!

I've managed several accounts with GCP over the years and I've always maintained a great relationship with our contacts there. Some of these accounts were quite small, on the order of <$20k/mo, and even then we were kept abreast of anything that might be cause for concern. I always maintain a standing biweekly meeting with at least someone on the other side (account exec, technical staff, whatever) and I've yet to be blindsided by anything.

Is Google's communication good? No, not particularly. The only way something like TFA happens is if the relationship is neglected (by one or both parties). I'm not saying Railway did something wrong, but there are usually many flags and opportunities to correct long before drastic actions.

I get the impression that Railway plays fast and loose with a lot of their limits and resources and that Google may not be a fan of that.

Edit: would also like to say that if you put all your resources in one GCP project you are going to have a bad time. If you organize stuff over many projects it is very unlikely that they will ever take account wide action. I've had issues with, for example, a particular tenant's behavior, but it never jeopardized the other tenants.

> what if that happens to a small account which hosts some really important data/services there?

Pray to @dang that you will make the front page of HN?

Even if you are a billion dollar company you still have problems like the Australian pension did. Google is just that bad.

https://blog.railway.com/p/series-b

Agreed. Railway are probably not far off a billion dollar company though!

I don't think you can ever trust one service with critical data. Some Claude instance deletes your prod database, you have to restore from an offsite backup because it also deleted your local backups. Even at small startups we did pg_dump to AWS from GCP because ... who knows what is going to happen to GCP, and we want to continue to be in business if that happens.

I don't feel safe with any one single point of failure. "Your credit card bounced", "you thought it was dev", "you got hacked", etc. are all the same problem to me and no cloud provider solves those merely by setting up an account.

Railway isnt far from being a billion dollar company, no ?

More here...

https://x.com/JustJake

After reading about how their delete database API also deletes all the backups, I concluded they are not to be trusted.

It's not back.

How is the title misleading?

I added the qs because it defaulted to a story from 3 months ago.

Nope, Railway was the company who was hosting PocketOS, which is the company that blamed Cursor for deleting their prod database. Railway is only involved insofar as their API allowed an instant delete of the prod database.

fairly certain you are remembering the goofy article that was going around where a railway user allowed an agent to delete his db. iirc he questioned the agent after and the agent told him it should have read the file that told him not to do things, so just sounds like he deleted his db and blamed his tools.

Yeah but until you find that the new cloud provider won't approve your compute quota or doesn't have enough capacity in the region or you hit fraud flags for stagnant account spinning up lots of compute.

How to handle domains? The rest is easy, but your domain registrar blocking you sounds like a pain. My current solution is to use a local small provider, just for the domain. Then if there is a problem with your play account it is out of any blast radius.

Same applies to all the companies betting the farm on AWS.