Hacker News

Really nice explanation of a useful pattern. I was surprised to discover that even the famously broken NFS honours atomicity of hardlink creation.

I tried using this a while back and found it was not widely available. You need coreutils version 9.1 or later for this, many distros do not ship this.

I made https://github.com/rubenvannieuwpoort/atomic-exchange for my usecase.

Title says Unix, renameat2 is Linux-only.

Which of these are not guaranteed by the POSIX specification? It’s been a while since I studied it, but if I recall correctly the ones mentioned in the article are guaranteed.

Yeah I agree that one isn't very clear, perhaps the idea is to use `msync()` as a barrier to achieve consistent ordering of the writes without having to handle that yourself with more complex primitives. But then, they do mention some of those primitives at the bottom of the article, so it's hard to say what exactly the idea is.

mmap/msync is behavior is also very platform specific. On some systems (like AIX, at least older versions), even without msync, memory mapped data is synced back to disk periodically.

I worked on a code base that was portable between Linux, AIX, and some other Unix flavors. mmap/msync was a source of bugs. Just imagine your system running for days, never syncing any data to disk... then someone pulls the plug. Where'd my data go? Even worse, it happened "in production" at a beta site. Fortunately we had a way to recover data from a log.

What are flocks in this context? Surely not a number of sheep...

In UNIX/POSIX file locks are advisory, not enforced, it only works if all processes play ball.

Aren’t flock and POSIX locks backed by totally different systems?

Windows had APIs for this sort of thing added in Vista, but they're now deprecating it "due to its complexity and various nuances which developers need to consider":

https://learn.microsoft.com/en-us/windows/win32/fileio/about...

In some cases, you can start by using the "at" functions (openat...) to work on a directory tree. If you have your logical "locking" done at the top-level of the tree, it might be a fine option.

In some other cases, I've used a pattern where I used a symlink to folders. The symlink is created, resolved or updated atomically, and all I need is eventual consistency.

That last case was to manage several APT repository indices. The indices were constantly updated to publish new testing or unstable releases of software and machines in the fleet were regularly fetching the repository index. The APT protocol and structure being a bit "dumb" (for better or worse) requires you to fetch files (many of them) in the reverse order they are created, which leads to obvious issues like the signature is updated only after the list of files is updated, or the list of files is created only after the list of packages is created.

Long story short, each update would create a new folder that's consistent, and a symlink points to the last created folder (to atomically replace the folder as it was not possible to swap them), and a small HTTP server would initiate a server side session when the first file is fetched and only return files from the same index list, and everything is eventually consistent, and we never get APT complaining about having signature or hash mismatches. The pivotal component was indeed the atomicity of having a symlink to deal with it, as the Java implementation didn't have access to a more modern "openat" syscall, relative to a specific folder.

I don't follow, sorry. Are you saying that if we run:

    mv a b
    mv c d

If that's not what you're saying, could you give an example of something you want to be able to do but can't?

As long as you don't run into or want freedom from possible path races, for that you need the missing:

  frenameat2(srcdirfd, srcfd, srcname, dstdirfd, dstfd, dstname)

This requires O_SYNC and O_DIRECT afaik.

Even then it is only some file systems that guarantee it and even then file size updating isn’t atomic afaik.

Not so sure about file size update being atomic in this case but fairly sure about the rest.

Matklad had some writing or video about this.

Also there is a tool called ALICE and authors of that tool have a white paper about this subject.

Also there was a blog post about how badger database fixed some issues around this problem.

Unsure. Aren’t there filesystems which make O_APPEND less durable than it’s specified to be, which might be interpreted to adversely affect atomicity? Could that be it?

The challenge with that approach is memory: trigger conditions, if added irresponsibly, can result in unbounded memory and (depending on implementation) potentially linear performance degradation of filesystem operations as well. Unbounded kernel memory growth leads to stability or security risks.

That tradeoff is at the root of why most notify APIs are either approximate (events can be dropped) or rigidly bounded by kernel settings that prevent truly arbitrary numbers of watches. fanotify and some implementations of kqueue are better at efficiently triggering large recursive watches, but that’s still just a mitigation on the underlying memory/performance tradeoffs, not a full solution.

Sounds half baked. What context does this function run in? Is it an interpreted language or an executable that you provide?

Inotify is the way to shovel these events out of the kernel, then userspace process rules apply. It's maybe not elegant from your pov, but it's simple.

I've used FUSE for something similar.

There are sample "drivers" in easily-modified python that are fast enough for casual use.

are you asking for if statements?

if(condition) {do the thing;}

incron

That's right, atomic operations work just fine for memory shared between processes. I have worked on a commercial product that used this everywhere.

I don't do devops/sysadmin anymore, so this would have been before the age of k8s for everything. But I once interviewed for a company hiring specifically because their deployment process lasted hours, and rollbacks even longer.

In the interview when they were describing this problem, I asked why the didn't just put all of the new release in a new dir, and use symlinks to roll forward and backwards as needed. They kind of froze and looked at each other and all had the same 'aha' moment. I ended up not being interested in taking the job, but they still made sure to thank me for the idea which I thought was nice.

Not that I'm a genius or anything, it's something I'd done previously for years, and I'm sure I learned it from someone else who'd been doing it for years. It's a very valid deployment mechanism IMO, of course depending on your architecture.

Why? What do you prefer to do instead?

why? it works and its super clever. Simple command instead some shit written in JS with docker trash

Works pretty well for Nix

Then you are locking yourself out of a pretty much ironclad (and extremely cost-effective) way of managing such things.

Isn't that the standard way to do that? Why wouldn't you?

that's how some phone OSes update the system (by having 2 read only fs)

that's how Chrome updates itself, but without the symlink part

Nobody's saying you should deploy code with this, but symlinks are a very common filesystem locking method.