Hacker News

Its a yes if you do not know the domain space, query patterns well enough and also if the cost of optimization or time for optimization may have detrimental impact to business. In this case it most likely means that the crowd in the room did not anticipate much on this in early phases and no one in the room pointed these things out. The irony is that these performance and disk numbers are heavily discussed as a part of system design interviews.

> In fact optimization is by far the easiest part of the process because there are many system programming experts on this HN thread who consider these optimizations to be trivia

This is a misconception when you including roll out as a part of the change too, changing data once its running in production is hard, changing the data structure is even harder and when you talk about making changes in cache which is at the hot path its probably the hardest. Looking at the graph at the end it looks like it took them 4+ months to roll out the changes after optimization.

I do not think Cloudflare was a less-than-peers optimized product when they launched. This is one of their blog posts which describes taking one aspect even further.

I think Cloudflare became big only because they were so much more optimized than others that they offered some services for free that others were not offering. If running costs are high, you only burn (VC) cash and then you exit.

Or optimize a bit earlier and prevent having to scale out to a bazillion systems.

This assumes that you have plenty of cash to burn in the process, which is approximately correct for VC-backed ventures, and for offshoots of large corporations that play a lomg game.

> start generating profit, and then you can start optimizing your costs

Good thing they jumped on that as soon as they were profitable instead of burning cash. Oh wait...

I think a distinction to draw here is that Cloudflare had relatively large capital raises and were almost immediately profitable¹. They had the luxury of throwing away money. Judicious optimisation makes sense for scrappy start-ups, especially when trivial optimisations like these could easily be farmed off to an agent.

¹ https://timeline.www.cloudflare.com/

This reasoning assumes you have access to infinite runway. You don't.

Only if you have loads of capital

> Produce working product first, validate the idea, stabilize the business, start generating profit,

not everybody is so lucky to be able to go in that order? The first part requires upfront capital/investment?

Rust does that automatically unless you switch to the C layout.

In langages that don’t there’s a tension between memory use and human readability / consistency of the layout. There are also other domains which can be affected e.g. databases, it’s a concern / issue when using postgres.

For the curious, this is technically possible in Rust using a dynamically sized type [1], but in practice is difficult and doesn't really play nice with the rest of the language. The nomicon entry concludes with "Yes, custom DSTs are a largely half-baked feature for now." [2]

[1] https://doc.rust-lang.org/reference/dynamically-sized-types....

[2] https://doc.rust-lang.org/nomicon/exotic-sizes.html

> putting the record data right after the CacheEntry members

I assumed they couldn't do that because they're using it with some kind of generic HashMap<K, V>. In that situation, can "V" be dynamically sized?

A dynamically sized "V" would mean you can't have an array of them, which might preclude some hash map implementations.

Unfortunately, Rust is not a good choice for this kind of tricks. This is where Zig shines. In Rust, you can’t even use proper arenas, which can help a ton with allocations.

Cloudflare started to pick Zig recently, for projects, that have memory constraints.

System programming always matters. Things are cheap until they aren't one day.

Depends on how the CacheEntry is stored, it's probably stored in a slice of &[CacheEntry] which precludes storing the record data alongside it as the size of each entry must be fixed.

I wish more programming languages implemented record types as seen in databases, where dynamically sized fields are packed into a contiguous area of memory.

The CloudFlare manually implemented a clumsy version of this.

Wouldn’t it be nice for the compiler to manage this for you in the same way that your database engine does when it saves a “row”?

less ergonomic, but still totally doable

Why do I always find interesting new Twitter accounts just as the person is leaving :)

I think it’s more of a time vs code tradeoff, if done properly.

For example in the Vec case, you could theoretically build an alternative which encodes the “three sections” property internally, and ensures correctness at construction time for the pointers. Not as completely safe as a Vec, but you can still get similar benefits for the “business logic”.

But I agree, just having a custom structure that does not provide a safe wrapper around this would be sacrificing standard guarantees.

You can make a wrapper type that abstracts the offset lookup logic with a safe interface. If it's a transparent struct then rust will compile it away into nothing but you still get the abstraction in your code.

> I can't help to think that the approach of joining several distinct list into a single one in some way undercuts Rust's safety guarantees.

Not really. You just need to make the underlying fields private and provide methods to get slices to the data you need.

you could always do a .get into the vector and handle the error, it doesn't necessarily need to panic.

Thank being said in this case it should be impossible to index out of bounds so maybe a panic is warented.

It's the exact thing Rust is made to protect against, on a more local scale. Every memory corruption bug is just an out-of-bounds index that wasn't protected against.

Tools exist to serve us, not the other way around.

AFAIK (at least it worked like that some 10 years ago) the captive portal just intercepts the HTTP page load and inserts its own content (most often a 302). So it just has to be a http web page. Firefox uses http://detectportal.firefox.com/canonical.html

Relevant support page, though light in details: https://support.mozilla.org/en-US/kb/captive-portal

Edit: ah, yes, DNS can be hijacked too (requires intercepting outgoing traffic on port 53 therefore incompatible with DoH), that may require fewer computing resources. Still need http otherwise the server cannot use the correct cert chain.

Edit 2: Wikipedia says both methods are used: https://en.wikipedia.org/wiki/Captive_portal and also mentions RFC 8910. I suspected something like that existed, hence my initial disclaimer.

My point was: that domain is not treated any differently from other domains.

Dumb captive portals, which do still exist in some places, usually do MitM attacks on the connection, so you need some http(no-s) site that you can abuse as "yeah, this can get attacked by the WiFi" to then answer the portal.

The right way is that there's DHCP option for the network to signal "I have a captive portal", that's been standardized for over a decade.

… or … IDK … just stop shoving ads down people's throats just because they want WiFi.

That’s a Mac bug if so—it should be always using dumb udp/53 for captive detection, not some fancy DoH thing.

this applies to more than DNS caches. In 1998 I mailed Microsoft a proposal to replace search engine crawlers with a push-based filesystem monitor (detect change → extract → compress → push to index). Got a 5-line rejection letter. They built the same thing 20 years later as IndexNow. Full story with the original letter: https://dev.to/andrew_vl/in-1998-i-proposed-push-based-searc...

It can be quite hard to predict where particular usage patterns will take a piece of software under extreme load, especially with things that have lots of internal state. Obviously when you get to spend 100 T or more the pay off of an optimization is much larger than what it is in the case of 1T or less, and your typical developer is not going to have that kind of memory even in aggregate to play with. I tend to be forgiving when it comes to watching software bloat that I did not cause myself (and yet, I'm frustrated that Ubuntu's start-up greeting message takes a whopping 500 M).

In the case of internet infrastructure I don't think there was anybody even up to the year 2000 who had any idea of how bit this was going to be. And even now we have IPV4 and lots of legacy to deal with. Cloudflare is not my favorite company, let's put it like that, but in this case they show how the sausage is made and I think that should be applauded. Much better than 'why were down again for X hours'.

That’s the old school of thought. These days, designers of newer serialization formats realize that designing a more compact format doesn’t really buy much on modern CPUs and modern networks. See for example Cap’n Proto (whose inventor, kentonv, also works at Cloudflare) and flatbuffers.

An approach like that would be at constant war with the borrow checker in Rust. Apparently it is possible but there is enough friction that these guys went a different route.

I would also have instinctively reached for a large VM reservation to exploit demand paging. I have used that pattern a lot in C++ but not in Rust, so I don't know how difficult it would be to implement there.

Rust supports punning via pointer casting, but you'll want to use #[repr(C)] on any data types used

It’s finally gaining some traction…

https://www.google.com/intl/en/ipv6/statistics.html

Currently $15 per GB, he saved Cloudflare $1,500,000 and got exactly $0 bonus. He must really believe in cloudflare's vision (global enshittification). In related news, three times today Cloudflare told me that I'm a bot and shall not pass - not that it needs to check if I'm a bot before it lets me pass.

Cloudflare talks about having datacenters in 300+ cities. Presumably they have at least a few servers per datacenter. They saved 130 servers worth of memory... not even the minimum number of servers they have (seriously though, they probably have a LOT of servers)... a few GBs of memory per server running the service. At that scale this is a nice-to-have.

probably agents going through tech debt or finding wins

every dept knows what they could do with more budget, the budget for those things just never comes

now agents have utilized budget more effeftively, unbottlenecking many things, including engineering blogs

This kind of encoding[0] is ubiquitous in networking protocols. It scales down to small silicon well and enables the receiver to estimate resource requirements or skip parts of a serial byte stream without storing it in memory first. These encodings usually aren't aligned by design.

[0] https://en.wikipedia.org/wiki/Type–length–value

It's called TLV encoding - tag/length/value. It's very common in all sorts of network protocols and serialisation formats. It allows you to skip unidentified tags. Sometimes, like in the PNG file format, there's a fixed bit in the tag that tells you whether it's safe to skip or if you have to reject the whole thing because you don't understand this tag.

Hey dang can I get my rate limit turned off pretty please?

DNS is important and 100TB of RAM is effectively nothing.

> If your cache is that big it isn't a cache.

This is an incorrect statement. Caches do not have a requirement of being smaller than their source data set. CDN is an example of a cache that generally matches the size of the source data.

Maybe I'm misunderstanding, but this powers 1.1.1.1, it doesn't front an internal dataset. A cache miss hits a nameserver. Which is to say, the dataset is "every DNS record in the world"

You have to cache, cloudflare doesn't know all the records ahead of time, they have to do recursive lookups to the authoritative servers that own the records and that is only good for the period of the TTL of the record. There is no "global" DNS record database or something like that.

It's a recursive resolver. The global DNS dataset is not something you could collect to serve directly vs caching from observations.

The data source is authoritative name servers operated by third parties, some of which are slow on their own, some of which are behind slow or lossy networks. Origin response times vary between probably 1 ms and 2 seconds +/- origins that never respond.

They’re adding the cache consumed across all of their servers. It’s not one giant deep cache.

The simple answer is that if you didn't cache, DNS traffic would skyrocket, and the load would pile up on the authoritative servers, which were intended to be small, and during the early days of the Internet, were frequently on bandwidth-constrained links.

DNS is designed to distribute query load to the edge as much as possible, and that's enabled by caching. It just so happens that "the edge" is now becoming concentrated among a small set of providers because they wanted to make a business out of it.[1] They knew that this would be expensive going in, though.

[1] Nobody has to use 8.8.8.8 or 1.1.1.1. Most people can use their ISP's cache or a local cache instead without any noticeable difference in behavior.

Rob Pikes 5 Rules of Programming:

Rule 1. You can't tell where a program is going to spend its time. Bottlenecks occur in surprising places, so don't try to second guess and put in a speed hack until you've proven that's where the bottleneck is.

Rule 2. Measure. Don't tune for speed until you've measured, and even then don't unless one part of the code overwhelms the rest.

Rule 3. Fancy algorithms are slow when n is small, and n is usually small. Fancy algorithms have big constants. Until you know that n is frequently going to be big, don't get fancy. (Even if n does get big, use Rule 2 first.)

Rule 4. Fancy algorithms are buggier than simple ones, and they're much harder to implement. Use simple algorithms as well as simple data structures.

Rule 5. Data dominates. If you've chosen the right data structures and organized things well, the algorithms will almost always be self-evident. Data structures, not algorithms, are central to programming.

https://web.archive.org/web/20260314210910/https://users.ece...

It is often not worth optimising in the early days. You don't know how popular it will become, you might not know how many DNS records you will hold, it was possibly written in an earlier language and ported as-is.

At the point someone queries the 100TB of RAM, then maybe it is worth revisiting but even that has risks. You have to design the migration path, have fallback mechanisms etc.

Premature optimization argument fits right in. Now that memory is up to 10x more expensive it is worth considering optimizing programs with large memory footprint.

Discussing trivial optimizations is a waste of valuable design time. You're never going to "forget" an optimization. The running system will remind you when the optimization is actually needed.

Boxed slice isn't really the most well known type/optimization, There usually aren't that many vec's that it makes a big difference.

it was working so no one thought to check