Hacker News

I am disappointed you're downvoted, Richard. This is a fine reply, and I hope you know that a minor quibble with a single line in the post doesn't mean that I think it's a bad one overall. (EDIT: a few minutes later, the parent comment is no longer grey.)

I also think it's a good thing that you wrote the post in general, when I saw it pop up I was like "oh, of course, this post should exist!" I'm surprised I didn't think about it earlier.

> evaluating userspace code at compile time

Usually this would be done via an interpreter, so I'm not sure that it really requires unsafe either. If you are literally executing machine code, sure, but const fn in Rust and constexpr in C++ and many other languages do not do that, as it causes a number of problems (for example, cross-compilation).

Because they view "unsafe" as an escape hatch instead of a feature. It's a way to encapsulate dangerous behavior, tightly, with clear postcondiitions. Sometimes it's the only way to do things like interact with inherently unsafe FFI code, or hardware.

> In extremely high performance code you use different data structures and algorithms and change your approach to memory allocation.

It's worth noting that the reason Rust doesn't include support for custom memory allocation patterns like Zig does has nothing to do with memory safety. It's more of a historical accident that it just wasn't something that was prioritised early in the projects history and is now hard to change.

I do think it reflects different priorities, but one of those differences is that from my perspective, safety and performance are not inherently at odds. Yes, sometimes it is needed, but not as much as some people seem to think. Sometimes, it also means writing code in ways that communicate things to the compiler that you may not think of if you're not used to thinking in this manner.

A lot of the ways in which the zig compiler works doesn't use pointers, it uses indices. This stuff is easier to write as safe code, not less easy.

> Roc is attempting to make a similar set of trade-offs in their compiler as Zig, so it makes sense that the author finds many shared patterns.

I do think that that makes sense, but it also doesn't mean that they have to. I am doing a compiler project that takes a lot of inspiration from Zig (as my language currently inherits some major things from Zig, and I also care a lot about compiler performance) and it's written in Rust, and does not use much unsafe code (outside of the usual suspects of FFI in the runtime, etc).

They are saying that running the compiled code is memory-unsafe when there is a compiler bug, and that’s what developers do next. The memory corruption happens in a different process.

In this respect, effectively all the compiler should be treated sort of like an unsafe region because it requires extra care to avoid memory corruption bugs.

In context that's clearly not what he's saying, the next sentence is this:

> Zig has more features than Rust for making memory-unsafe code work correctly, and that was the area where we wanted the most help.

Zig definitely does not have more features for successfully emitting memory-unsafe machine code than Rust does. I can emit memory-unsafe machine code from typescript if I really want to and nothing at all in the language will get in my way. So the sentence quoted above must refer to the idea that the compiler itself needs to be unsafe, which Steve is right is simply untrue.

I do think that is a good point, it's just not what the line actually says. But that's why I wasn't saying "zomg this is WRONG!!!!" but instead, trying to point out that there are subtleties here. For people who aren't as deep in the weeds in this subject, I think the details matter. But again, as I said, I like the post, this is just one thing.

I am also probably in a more pedantic mindset because, well, I'm writing a compiler in Rust, and the words as written do not resonate with me at all.

> a perfectly safe compiler that generates vulnerable binaries isn't that much better.

I do think it's much better. Eliminating classes of bugs in one component is a good thing, even if it's not every component. This is a core lesson of Rust! unsafe still exists, but going from "I don't know what is unsafe" to "only this part is unsafe" is a major improvement.

The section this came from was talking specifically about usages of `unsafe` in the compiler code.

It's not about the memory safety of the resulting binary.

> probably be a linker

I don't think that's any different either. The core job of linking isn't particularly unsafe.

(Unless, similarly, you're doing the hot reloading stuff)

Maybe in theory. In practise you have thing like super optimisers. You have side effects that the compiler needs to understand etc.

That said I'm struggling to think of something that would need to be unsafe.

I don't know enough about Zig to explain it, but there is more to ReleaseSafe than checks and panics. ReleaseSafe also clears memory that no longer has an owner (I might be describing that wrong, that is just how I understand it). I found this out with a rendering issue recently.

The bug was around passing a slice to OpenGL which referenced memory outside of its lifetime. Since the memory location had no owner, vertices would still exist in Dev builds and everything would work fine, but in ReleaseSafe the application would run and just have nothing to render.

This is the commit where I fixed the issue: https://github.com/quot/donut/commit/8fff107e76278c4bf55007c...

The DebugAllocator catches use-after-free (at least on page-level), but at the cost of never recycling memory addresses (e.g. it eats through the virtual address space).

https://ziglang.org/documentation/master/std/#src/std/heap/d...

For higher level code, "generation-counted index handles" might be the better solution to provide temporal runtime memory safety, not part of Zig the stdlib though.

Or even better: never use dynamic memory allocation and make all lifetimes 'static' :)

I appreciate the insight, and on closer reading the post clearly states that realistically only Zig and Rust were ever considered anyway.

Since you're here, could you comment on the approach Rust took in their rewrite? Was it more of a straight translation like Go did when they self hosted -- similar to the recent Bun transliteration? Or were there architectural changes made along the way like this article describes with Roc?

OCaml compiler is incredibly fast. I wonder how it'd fare with Jane Street's extensions for the borrow checker etc in OxCaml, if it's good enough for their HFT I'm sure it's good enough for a new language.

Depends on the beholder.

Unix system programming in OCaml

https://ocaml.github.io/ocamlunix/

https://mirage.io/

OCaml has often historically been considered a language that's been appropriate to write systems tooling like compilers, runtimes, and unikernels in, even though GC'd languages were/are not often considered for such projects.

I'm pretty sure Zig has no plans to ever become safe - by any sane sense of the word - so, yes, I would expect...

Chief design goals were radically easy concurrency and speed of compilation.

Is the Go GC that special? Is it even generational yet?

Goroutines?

It's literally the most sophisticated scheduling engine in the world.

In practice, Go can typically outperform Rust in throughput (using more memory), despite having a mountain of disadvantages against it in theory.

That's how good the Go scheduler/runtime is.

That's sort of what I'm doing...

I'm writing a language with Affine Ownership that transpiles to Zig and has a built-in FSM-based Green Fiber runtime.

Affine Ownership gives you memory safety + fearless concurrency + eliminates the need for Go's GC.

It's obviously going to slow down compilation - since you need to do Rust's borrow checking, etc. But I can do this incrementally as well...

> how about from the other direction, adding some kind of borrow checker to Zig? That sounds more within reach and practically achievable, possibly even in userland.

It's doable, and as static analysis. see sibling comment.

It's impossible to add a borrow checker to any existing language.

The reason Rust has a working borrow checker is because every part of the language from structs, enum, traits, generics and all the way to the syntax itself has been designed to support lifetimes and borrow checking.

It's is not something you can just tack on to an existing language without fundamentally changing it.

> Compilers are not security sensitive, usually.

The compiler is one of the most significant trust boundaries we have. Its decisions can intentionally or unintentionally create vulnerabilities in programs compiled by the compiler, which means that if you can compromise a compiler you can compromise everything downstream.

Unsafe memory access in a compiler can be exploited in order to hijack the compiler itself (this is reported regularly in production compilers), allowing the attacker to then insert arbitrary code into compiled binaries. Not everything that a compiler absorbs from its environment is meant to be treated as source to be compiled, and in a memory unsafe compiler any of that input can silently turn into machine code in the compiled binary if an attacker is able to exploit the memory safety bug and hijack the compiler.

Of course they are, anything can be a gateway to inject backdoors, if security is not taken into account.

And as mentioned, if what Zig offers is already in Purify, there is hardly any added value over C and C++, without the headaches of a niche language.

Sounds like what I was thinking, that or for third party deps to go into the same temp folder, and anything not accessed in over a week, or so just gets auto wiped by rustc / cargo build process?

Can you talk a little more about that? How would that work, and what would the developer experience be like when using it?

Rust isn't great, and it shouldn't be a surprised since it's designed after npm. However one metric where nodes_modules is still worse for me is the sheer number of small files in it.

Having nearly one million files in nodes_modules isn't that unusual. The problem is that on most common file systems the minimum allocation is usually at least 4KB. So even if the actual data is less than 500MB, you end up with 4GB disk space used/wasted.

> Zig is not pre-1.0 because it’s not ready for production (bugs or missing features), it’s pre-1.0 because they want to be able to make breaking language changes.

This is a solved problem in other projects. Either use the version numbers as intended and bump the major version number on breaking changes, or use Rust-style editions to opt in to the newer versions of the changes.

Calling a project production-ready but keeping the version number below 1.0 and saying breaking changes are expected is a tired game. We've seen it backfire across a number of language projects like Elm, where the exact same claim was used to both encourage people to use it and then blame them when it backfired.

If it's production ready, go to 1.0 and then follow semver for breaking changes. I don't care if we get to Zig v73.2.0 as a result. At least we can see from a glance which versions need to be checked for breaking changes.

> Nowadays when you can just point an agent at release notes and have it update everything

Except that means that not only you lose compiler bugfixes, you also pretty much has no access to the ecosystem. For most production codebases, this is a deal breaker.

> they want to be able to make breaking language changes

That sounds like it's not ready for production to me.

Makes sense. Thanks!

Yeah I get they use it but I don't understand why you would buy it. it's just the runtime for code that makes up the agent.

I work both on a pretty much bog-standard web (GraphQL) backend and the frontend that uses it. We switched over from Apollo on node to async-graphql on Rust.

The runtime performance is much better, but the compiler time performance is terrible. To be fair, this is mostly the fault of async-graphql, but that doesn't really matter all that much. For example, it's not uncommon for a single character SQL query change to trigger over a minute long incremental rebuild.

The rust compiler is just choking on the number of generics and codegenned functions.

I've personally looked at how to improve this, but short of breaking up the type graph using federation, nothing can help. Not even cranelift makes a noticeable dent.

Additionally, the team started off composed by a bunch of TypeScript/React/Node developers, so mistakes were made along the way.

Honestly, I would have recommended to just use C#.

That's not to say that I don't think Rust can work for web development. We have some (GraphQL-less) services where Rust is a great fit. Just maybe shouldn't have been the default. That or give up graphql ...