Fearless SIMD v1.0 (linebender.org)

122 points by verdagon 2 days ago

modulovalue 3 hours ago

I’m currently working on adding better SIMD support to Dart (https://github.com/dart-lang/sdk/issues/64170) and I have a question for the author or others here.

Does Rust or any other language support customizing the compiler so that interprocedural analyses can track custom subsets of, for example, doubles so that the compiler can choose the most efficient instruction sequence for example for min/max? If we know a double is never NaN then we can emit only one instruction on x86, but have to emit one more on arm64. If we know a double is never zero and never NaN, we can emit a single instruction on both.

This whole conversation between relaxed SIMD and deterministic SIMD seems to only exist because our compilers are not smart enough and/or their whole program analyses don’t support any plugin-like capabilities.

There are other examples where if we know a SIMD bitmask is canonical (all 1s per lane) then we can implement horizontal reductions more efficiently. This is very niche and I doubt that any language supports interprocedural analyses with such a rich domain, so it feels like a hole in the programming language space.

JonChesterfield 3 minutes ago

Customized compiler isn't really a thing in most languages. Because it implies changing the language, and everyone agreeing on what the language is was sort of the point of the whole exercise. Hence all the worries about macros.

SBCL has the right hooks. Jai probably does as well. You could build your own thing on LLVM relatively easily - so Rust _could_ do it if you gave it sufficient access to the compiler, but so could C++ or D.

SIMD bitmasks being ~0 instead of (00000001)+ is common and useful, but there's an annoying language design question in there about what type comparisons between vectors should be (if you don't have a <N x i1> as a type, say because you liked C too much). Shout out to std::vector<bool>.

There is probably interesting work in mojo for this. The library being in MLIR strongly encourages taking that sort of approach. I haven't looked at their implementation though. Happy hacking!

VorpalWay 3 hours ago

Not yet that I know of, though perhaps LLVM might be able to infer simple cases (when loading from a known constant for example).

Pattern types could potentially maybe in the future allow for the compiler to know more details though. They are a nightly feature, and afaik only for enums, integers and pointers so far. The idea would be that you can define a custom type such as "an integer between 7 and 45" and everything else become niches for niche optimisation (e.g. for `Option<MyFunkyInt>` some of those impossible values would be used to represent the None case of the wrapping Option).

But I could envisage a future in which you could say "f64 without NaN" which would both make those available for niches and potentially tell LLVM about this. However, we are very far from any of that currently. And it might not be what you want, since you would need to add checks when you perform operations to ensure the value doesn't suddenly become a NaN. Which is way more complicated than ensuring integers don't become, say, zero. It will likely be much harder to optimise away the checks.

borodi 2 hours ago

LLVM has range flags and things like nnan that could in theory be used to replace a minimum intrinsic into a minimumnum (iirc x86 has a instruction for the latter but not the former) https://llvm.org/docs/LangRef.html#floating-point-min-max-in... I'm not sure if the optimizations use this but in theory they could

modulovalue 2 hours ago

> Pattern types could potentially maybe in the future allow for the compiler to know more details though.

Thanks, I’ll take a look!

VorpalWay 2 hours ago

athrowaway3z 3 hours ago

Last time i checked; no.

But I suspect you're overvaluing the potential savings. Knowing when a float is 0.0 or NaN beforehand is almost entirely impossible, except for the most trivial of cases - like when you first initialize a variable or first enter a loop. Everything after that is very hard or impossible with floats as they are.

Those cases can be const folded at compile time.

Those cases are never a measurable bottleneck.

The closest thing I know of in the realm of the optimization you're curious about is Rust NonZero* variants, but they're used for enum compression afaik.

modulovalue 3 hours ago

I’m not sure I agree on the impossible part, I feel like a sufficiently smart interprocedural analysis that also implements range analysis interprocedurally could prove a lot to where it becomes useful.

I guess what I would like to see is SIMD libraries being able to confidently say nobody needs to use intrinsics (or differentiate between relaxed/normal SIMD on the user API level) because the language + high level SIMD APIs are smart enough to choose the right implementation.

IIRC IEEE min/max with proper NaN handling needs 8 instructions on x86 vs 1 on arm64 I find it very sad that we apparently haven’t really solved that yet without forcing the user to use different APIs.

athrowaway3z an hour ago

orlp an hour ago

Dr_Emann 43 minutes ago

Have been really happy with Fearless simd, I used it to write [memchr-n][1], which searches for instances of an arbitrary set of bytes, with simd, which actually tends to [outperform][2] the rust memchr crate, even for sets of 1-3 bytes (what's supported by memchr)

[1]: https://github.com/Dr-Emann/memchr_n#performance [2]: https://github.com/BurntSushi/memchr/pull/241

dang 4 hours ago

Related. Others?

Towards fearless SIMD, 7 years later - https://news.ycombinator.com/item?id=43519823 - March 2025 (175 comments)

Towards fearless SIMD - https://news.ycombinator.com/item?id=18293209 - Oct 2018 (81 comments)

nnevatie 2 hours ago

How does the crate’s SIMD performance compare to ISPC, which already has very easy dynamic dispatching?

PoignardAzur 2 days ago

Pretty happy this broke the 0.x curse!

the__alchemist 3 hours ago

The default system for Rust libs is ZeroVer: https://0ver.org/

meowface 3 hours ago

PuTTY out since 1999 and bumped to 0.82 in 2024 is pretty incredible

pocksuppet 2 hours ago

It's exactly what we were doing before the semver hype cycle, but with 0. in front!

JMKH42 3 hours ago

I started an early Rust SIMD crate with similar aims (SIMDeez) and so I know how hard it is to do this well, so wanted to say congratulations to everyone who worked on Fearless SIMD. Tools like this are really nice for being able to leverage the gigantic performance modern CPUs make available without having to write intrinsics for each platform.

elendilm 3 hours ago

Congratulations.

Nice work. Looking forward to it.