Hacker News

Even more so, it shows that SoA data structure means you can add fields to your 1M monsters with little impact.

Every Struct Matters

> Every object (i.e. not a primitive) has a header that IIRC is 12 bytes. But there is good news in JVM land: this will be reduced to 8 bytes in the next JVM release

Since JDK 25 it's already 64 bits with the `-XX:+UseCompactObjectHeaders` flag [1], but in JDK 27 it will be the default [2].

> where it requires too much heap to compete with the AOT compiled languages

Not to compete but to beat, and not too much, but the right amount. Low level languages are optimised for control, not performance (that control translates to better performance in smaller programs, and to worse performance in larger programs), and their particular constraints prevent them from enjoying certain important optimisations, especially those offered by JIT compilation and moving collectors, which remove some overheads that AOT compilers and free-list allocators incur. Their memory management is forced (by their constraints) to optimise for footprint rather than speed.

There are common misunderstandings about memory management and why moving collectors were created to reduce the CPU overheads of malloc/free, especially in large programs, in exchange for what is effectively free RAM. This is why moving collectors are chosen by the languages that are unconstrained enough to use them and have the resources to implement them (Java, .NET, V8). With the exception of Zig (and even there it requires some effort), it's hard for low level languages to use the basic optimisation that's behind moving collectors. I gave a talk about how moving collectors optimise memory management at the last Java One, and it should be available on YouTube soonish [3].

> but its startup time is too slow compared to interpreted languages

That hasn't been the case for some time. You are right, though, that startup/warmup time is worse than in AOT compiled languages, and that is the tradeoff of optimising JITs: reduce the overheads associated with AOT compilation in large program in exchange for warmup.

Both startup and warmup have already been improved thanks to Project Leyden's "AOT cache" [4], but it will never be as low as C.

In general, the tradeoff is between optimisations that help large programs vs optimisations that help small programs.

[1]: https://openjdk.org/jeps/519

[2]: https://openjdk.org/jeps/534

[3]: I can't reproduce the full talk (which goes into the maths of memory management) here but what happened with moving collectors was that until very recently (open source low-latency moving collectors are newer than ChatGPT), they required pauses and so weren't suitable for programs requiring low latencies. As a result, many developers either forgot or never learnt just how incredibly efficient moving collectors are. But the key is that because accessing RAM by necessity requires CPU, using CPU effectively captures RAM even it's not used by the program. Bringing the CPU and RAM usage into a good balance is more efficient than trying to minimise one or the other. This is also the reason why hardware (physical or virtual) is packaged within a very narrow band of RAM/core ratio.

[4]: https://www.youtube.com/watch

Most of real world use of Java platform has next to 0 concerns like those. Some more niche use case may benefit, good, but overall success map isn't changing anytime soon. Reasons for its long term success lie elsewhere.

> Most developers, in Java and in most other languages, do not consider the cost of every field

Are you saying most developers are bad? It’s the equivalent of most employees don’t consider the cost of every action to the employer and is how company spend blows up.

How do you deal with all the daemons and automatic crap that does this on Mac? Isnt it all reinforced by SIP?

If you have hot loops with millions of iterations at a time, structure your code accordingly. Its not anti-OO to choose the right data structure for the job.

And avoid moving said data between physical threads as much as possible.

Most of the bottlenecks I see are not due to the organization of data. Unnecessary communication of data is the #1 offender.

Maybe someone can write an OO language where arrays of structs are automatically stored as structs of arrays.

mild /s

It doesn't quite make sense to keep object identity at the language level. Inherently the data in the arrays cannot be the same memory of the data in the objects fields.

To get the speed up, you can't just abstract it as an access pattern because it's tied to the specific way the memory is laid out.

If you were trying to make some kind of collection type that could be queried by both row and column, you would need to store it both ways at all times and also keep both representations in sync, which also defeats the purpose, somewhat.

I feel like if you're trying to do this pattern then it doesn't make sense to also keep the objects.

We'll get there, bit by bit.

We need an ending to byte-sizeism as well.

In combination with “What colour are your bits” I do not see this ending well..

Add it to my watch list!

CppCon 2014: Mike Acton "Data-Oriented Design and C++"

Andrew Kelley: A Practical Guide to Applying Data Oriented Design (DoD)

you should check these two talks out then.

They will absolutely do that (prefetching, they can even eagerly load what’s on the other side of a pointer).

However it requires additional hardware to recognize patterns which benefit from prefetching, and every time the CPU prefetches data which ends up not being used it has both burned energy and memory bandwidth, and evicted data from the cache which might be needed (cache pollution).

A cache line is simply the unit of data a CPU cache works with (generally 64 bytes, because someone somewhere has probably determined that that is the best line size for general use), much like there are units of data like bytes (8 bits nowadays, but there have been weird ones historically), pages (varies between hardware as well, and may be OS-configurable), etc.

As TFA mentions, a CPU does some predictions about what cache lines to prefetch, e.g. when you do sequential reads. Moreover, the x86_64 instruction set provides a prefetch instruction through which you are able to give the CPU a hint "hey, I'm gonna be using this soon, prepare accordingly, pretty please".

Still, the utility of prefetching is diminished if you only use a single byte from each cache line, because the mechanism generally depends on you doing other work while the next cache line is being fetched. So really the best case scenario is to take as much time as possible to work with what is already fetched, so that there is time for the next unit of data to be fetched in the meantime.

It might sometimes prefetch the surrounding lines as well, but ultimately cache space is limited, so there is a trade-off. Every time you fill a line, you are throwing away something else that was cached there previously, which you may need again in the near future.