Hacker News

For people in within the industry or the tech side of it, Netflix’s engineering blog has always been fascinating and extremely useful because of the insane amount of stuff in this space they have solved or reworked. They have put more into tech side of modern-day TV/film than anybody else, and it's not even close. In a technical/workflow sense, working on a Netflix show is unlike working on any other. I have my issues with Netflix in other respects, but with respect to technology and workflow, they are awesome.

If you’re unable to appreciate a behind-the-scenes look at their engineering because the technology isn't for you or available to you, that's totally valid! But it's a you're not interested thing, not a Netflix is boasting about something that doesn't matter thing. Only a few thousand teams in the world need most of what they do over there, but that doesn't mean they aren't massive technical achievements. Most of them are. The scale, complexity, and cadence of modern production has given rise to some of the biggest technical challenges I’ve ever seen. And for anyone close to that world, this kind of content is of great interest — if not genuinely valuable.

I’ve sat through a few Netflix talks and they’re all the same flavor of “look what you can achieve with millions of dollars and hundreds of engineers.” They’re somewhat interesting from an architectural perspective, but even scaled down versions aren’t feasible in most environments and it leaves a taste in your mouth that you just sat through a recruiting pitch.

The purpose of these articles is to promote the brand among engineers, help hire engineers, and help the careers of the authors both internally ("I'm helping the company hire") and externally (you can point at what you built because it's now public).

Ha! Yes sure, but it does make sense that they’d keep something like this to themselves.

Creating massive amounts of high quality content efficiently, on a global scale, with seamless global distribution is an incredible competitive advantage.

I don’t see why they would provide it to anyone outside of their ecosystem.

It’ll be interesting to see if they translate this to games as well.

I know quite a few people who worked on this and unfortunately this is effectively the product.

It’s a company that prioritizes micro services and enterprise style crud apps internally. I’ve seen so many of their presentations and it’s like an IBM demo.

It’s data , data, data. That’s their approach to everything.

Hi there, I'm a fellow filmmaker building my own tools and would love to hear your thoughts and needs. If you'd like (and anyone reading this), please email

Studios at weedonandscott dot com

Yeah, I had the same feeling. The tech itself sounds genuinely impressive, but the article reads more like an internal case study than something aimed at engaging the wider community

For your first point, you can see it in the videos they have embedded.

I've worked at a few different FAANG's (including this one). I've participated in some of these engineering blog posts.

They will never tell you anything that is real-world relevant for you.

At best, you might get some kind of _theoretical_ insight. It's because they're operating at a scale that just isn't realistic for hobby developers.

But they're still engineers just the same as you and me. So they write blogs like this. And it's interesting! I love to read them.

> we were backing up, two or three times

So they just rediscovered what IT world knew for decades, or what am I missing?

Back then, it was basically the Wild West with digital cinematography. No wonder studios freaked out at the idea of lost footage

Weren't the Star Wars prequels the first big digital productions?

> large budget feature

What does this mean?

Collateral was not the first fully digitally shot feature film. In fact, Collateral was not even fully digital. (The first major, all-digital, HD feature film was Attack of the Clones, but there were other fully-digital feature films before that, just not as major, and/or not always HD. Robert Rodriguez' Once Upon a Time in Mexico (2001) was fully digital.)

But you are right that Collateral did do something very new/unusual at the time, and that was shooting scenes in higher frame-rates than 24, and mixing multiple frame rates in a film. (This might not sound like much, but until this time, pretty much every film was 24fps for the previous eighty years and it had a very specific look that everyone's eyes/brains were conditioned for, unbeknownst to them.)

And the other thing that was very interesting thing about it (though not something very visible to a viewer) was that it was shot on the Thomson Viper FilmStream camera[1], which was the first major attempt at shooting not just digital, but very close to "raw". It was also a huge pain in the ass. The camera itself was massive, but due to the bandwidth, it recorded to an external storage array that had to be pushed alongside it at all times, and that was itself about the size of a shopping cart. (This device was hilariously referred to as the "Director's Friend.")

In 2002, my friend and I, both cinema nerds in high school, drove an hour away to the nearest theater showing a film called Russian Ark[2]. Why were journeying to to see a strange little Russian film where a never-named character walks the viewer through Russian history? Because just like each episode of the recently-released Netflix show Adolescence, this entire film was a single, very long, very complicated, unbroken shot. One shot. No trickery, no cuts that were just hidden to the audience, one shot, through streets, buildings, snow, ballrooms with a couple hundred choreographed actors, it was crazy. This is easy now compared to how it was back then.

As we've now established with Collateral (and this film predates it by 2 years), digital cinematography existed, but the storage was a real problem, the power was a real problem. Since this film was one shot, it needed almost 100 minutes of both, unbroken. And since it was a very complex moving shot, it had to be operated handheld. So essentially they had an incredibly ripped director of photography who operated the camera on a steadicam the whole time while a giant array of daisychained batteries and hard drives were lugged behind him. And they did it something like 100 times until they had a few takes where there were no mistakes.

None of this really means anything to anyone anymore, but at the time, to cinematography nerds at least, this stuff was all absolutely insane!

[1] https://web.archive.org/web/20040103133953/http://www.thomso...

[2]: https://en.wikipedia.org/wiki/Russian_Ark

This. “the medium is the message” Marshall McLuhan.

The actual content is invariably a result of the infrastructure behind it. And it’s not just the image, it’s also what kind of scripts are written, what kind of audience insights are passed on to creative producers, what kind of creative teams are selected.

Is it proving short-sighted? If you’re optimizing for cinematic art, then yeah. But they’re optimizing for subscriptions and global reach. That vertical will likely move to live-streaming, sports and other forms that retain subscribers. And on multiple global markets at the same time (not just U.S.)

It’s a weird vertical, they’re quite sophisticated in their approach, but it’s surprising how they sometimes contract entire chunks out. I’ve read academic papers talking about how Netflix is a very strange disjointed thing.

"Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway." - Andrew S. Tanenbaum, Computer Networks, 3rd ed., p. 83. (paraphrasing Dr. Warren Jackson, Director, University of Toronto Computing Services (UTCS) circa 1985)

We had to transfer a few 10GBs, if not 100GBs between Europe and the US back 15yo.

Bandwidth was of 100KB/sec at most, I suggested to do that fly over things if the systems team didn't want to raise the priority of that transfer, after prod tried 3 times over the weekend, sadly, they changed the priority of that flow, it took still like 40h? For the initial load.

Mailing hard drives of LARGE amounts of data was relatively common as recently as the mid 2010s.

How much global infrastructure has improved… but also how physical logistics can still beat the internet when you're dealing with massive datasets.

Wouldn't surprise me if they had something like this in a few cities with big production teams. There must've been pipelines in place from when 35mm was the standard for movies, and crews needed to get hours of footage developed and over to various people for review each day.

https://aws.amazon.com/blogs/aws/new-physical-aws-data-trans...

A single file isn't 100TB - an entire production of OCF is ~200TB. A single take is typically 10-200GB depending on a variety of factors.

The time and stress saved for users.

Netflix doesn't make "stories" or "films" or "art" or anything that impressive.

They make "content", and the distinction is super important.

> But it still wouldn’t address the real bottleneck: compelling storytelling.

The problem is, storytelling is risky. Either you stick to something bland (adaptations of popular books, cartoons or videogames) and it usually gives decent returns, or you go for something completely original - at the risk of it either going boom or going bust.

As your average cinema movie is a triple-digit million dollar business these days just in pure production and actor cost and double that in promo cost, it's hard to find banks to finance the production, and so the banks prefer to go with something "proven and bland" over something risky.

There is still a lot of manual, error-prone work, even at Netflix. Netflix just has workflows that ensure so many passes are of each task are done, with so many failsafes, fallbacks, tests, checks (both human and automated), and they start doing them all earlier in the production process than most other studios so they can handle things they still didn't account for.

Yes, it's pretty vertical, but essentially every big streaming platform or production company is. There's a very specific Netflix way that governs how all Netflix shows are produced. There is a very different but still meticulously standardized way that governs how all Hulu shows are produced, one for all Warner Bros. shows, etc. This is an area where being vertically integrated is totally fine. It not only makes enormous sense for these studios, but nobody outside of those environments wants or needs these workflows. Netflix's workflows are there to aid Netflix even more than their shows, and while there's a lot of excellent stuff in their workflows that most productions should utilize for efficiency/safety/whatever, there's also a ton of stuff that would make no sense to use independently.

I know some of the folks who worked on this and a big chunk is Java with some Python afaik.

Probably not. One reason is that this is a competitive advantage for Netflix but I suspect the larger reason is that they created this to optimize their primary business, while operating it for other productions would be a secondary business and not mission critical. Another reason is that productions not on Netflix have other commercial options for camera to cloud production workflows (Frame.io for example).

This workflow and tool set is tuned for Netflix and is probably opinionated in a variety of ways to conform with Netflix production standards and requirements. If your production is being funded by Netflix then you're incentivized to learn and use their provided tools.

I've seen people in healthcare send patient data through whatsapp to other doctors in the same hospital.

It was not allowed and they had reliable corporate tools to exchange patient data, but the UX probably felt so cumbersome they'd rather face legal risks than doing it the right way.

The only part of that that seems crazy to me is the Gmail part. I’d want to control the mail server.

This is a sign they need deep reform and good management. Many such cases in technology these days.

It's amazing how often critical workflows rely on duct tape and hope

I barely even remember the last one that I watched let alone enjoyed. I guess it was Arcane which is a total fluke.

Using faster, easier or cheaper production workflows aren't significantly correlated with end product quality, other than perhaps in the obvious sense that investing a large amount of money/effort into a production might cause the investing party to take more care to ensure ROI. However, there are so many counter examples of very expensive, high-effort productions lacking artistic merit that the correlation is weak at best.

Thinking about this, and the reasonable argument below that Netflix have also produced a number of prestige films and series that are genuinely great, I wonder if the production pipeline has a side effect: flattening the quality signal.

That is, it used to be (80s/90s) a lot more obvious what the prestige/not prestige boundary was. Cheap TV content (soaps etc) was shot on video, expensive content shot on film. Now everything looks the same. Perhaps the one remaining effort signal was lighting, but Netflix seem to have chosen very flat bright lighting styles for everything now. Bad news for us chiaroscuro lovers. And even when directors do try to do that, they've often over-estimated the HDR so you get the opposite: an entire series which is too dark.

Yes, strange to have so much and still feel like you'e lost something.

I remember reading once how all Netflix content is really meant for phones or smaller screens. Simple shots, not much background detail, lots of face closeups etc.

I remember when being made by Netflix was a unique and cool thing, it didn’t last long until it meant probably-slop.

Having worked on both quality and junk film productions I assure you the editing workflow is not the determinant of artistic quality. No film or TV program has ever been improved by the editor(s) trying to build their own NAS or hack a version control system together.

Are you serious? Have you watched Adolescence? It's got more soul and artistic merit than practically anything else I've ever seen. And that was just last week.

Maid? The Queen's Gambit? Baby Reindeer? The Crown? Ripley? BoJack Horseman?

Sure they make a lot of schlock too, because they're a business and that's what most of their audience wants.

But I don't see how you could possibly criticize them for that when they continue to put out some pretty astonishingly artistic and soulful stuff.

Lincoln Lawyer?

Some C-levels have gone for the "quantity has a quality all of its own" philosophy of media production.

My personal experience with netflix has been that a good filter for 'quality' is what specific TV series and documentaries various 'scene' groups in the warez/torrent community consider worth ripping and properly encoding.

There's a certain amount of manual effort that's required to properly encode a ripped netflix or amazon prime series. People who do this strictly for street cred in the piracy community generally don't waste their time on schlock.

I mean if we go by volume of awards nominations it seems like they're fine in the artistic merit department

This is exactly how I'd imagine the dreck on netflix is made :).

"Don't be snarky."

https://news.ycombinator.com/newsguidelines.html

[flagged]

A NAS with ffmpeg :D

To be fair, it's a really, really big NAS with an interesting use case.