Hacker News

I would probably dig into the reasons for the differences in the benefit on the test machine and in prod

I had an issue like this for optimizing pgrust. I had an optimization that showed no impact on my test machine (c8g.4xl) and showed a 20% improvement when ran on my mac. It turns out the issue was the instruction cache on the c8g.4xl was being saturated on the test machine but not on my laptop, moving the bottleneck to a different place

If you can consistently reproduce the performance difference, you're already half way there

Optimizing a multi-layer cache system is a Complex System task with no subset you can reduce to a simpler version of the problem. PG has a 3 layer cache--database, OS, storage--and when you combine that with concurrency and locality, you have a giant 5 dimensional space to map out. Building the right set of benchmarks for PG with guidance for the most important parts to navigate fro that space has kept me busy for 20 years now.

On the bright side it could probably run for years without hitting the mistake as well. But it is nice to get it out of there.

Fuzzers are brilliant at this and produce all kinds of insane floating point inputs.

When I got early access to pgrust my first question was whether it had been tested for building read-only standby analytics questions. The problem has goals beyond that, but having a real analytics node available just by playing WAL data from your primary makes pgrust a low risk eval for your data. Just have to build confidence the results are connection, which is being applied by all the PG infrastructure regression tests the code has been trained on,

You can try it. We're happy to help you with it, but expect there to be issues to work through. You would want to do it for something non-critical

Thanks!

> A question on 20s postgresql time - It does not look like you are accounting for reading data from disk

I choose the data size so that it would fit in memory on the machine I was testing on. fwiw, there's still a ton of overhead Postgres has that the toy example does not. For example Postgres will serialize the numbers into tuples and need to deserialize them to execute the query. That's why it's not an apples-to-apples comparison

pgrcolumnar is not the default storage method. Right now, it's exposed as a table access method. There's lots of design space for how to do this so I want to avoid pre-committing to anything

I want to build the best database possible. While Postgres is great, there are a lot of core issues that have been around for over a decade. We're working hard to get pgrust production-ready, and it will definitely be production-ready in the near future. I wouldn't be putting hundreds of thousands of dollars into this project if I didn't think we could build a production-ready database.

It takes years to test it thoroughly in real environments. You don't want your data to be silently corrupted.

potemkin industry. jeez.

That's a pretty gloomy view

I cloned it. The 6000 commits are the problem, not the defense.

5,940 commits, one author, 29 days. 5,067 of them, 85%, have a `Co-Authored-By: Claude` trailer. Busiest day is 1,393 commits, running 60-105/hour for ~20 hours straight. Commit messages reference .claude/skills/fleet/ and agent worktree "lanes". That's an unattended agent fleet committing once a minute around the clock. Commit count used to mean review time. Here it means GPU time.

Also: the v0.2 tag shares no common ancestor with main. git merge-base fails. "The actual git history" is an orphan branch grafted in after the fact.

And the testing rigor we're crediting them for should be weighed against the headline claim: 300x faster than Postgres, ahead of ClickHouse: fastest analytical DB on earth, one guy, one month.

Most damning: the Postgres baseline ran with *max_parallel_workers_per_gather = 0*. Parallel engine vs. deliberately-hobbled single-core Postgres, headline says 300x.

Using AI is fine. But "look how many commits" is precisely the signal this workflow is built to fake.

No, it doesn't. Different algorithms can vary pretty wildly in performance based on the design of the hardware they run on. For instance cache sizes can make one implementation of a sort on a certain sized dataset faster or slower than another. You can have a theoretically fast algorithm that just isn't as cache efficient as a theoretically slower one in big-O terms. All levels of the memory hierarchy as well as storage have specific bandwidths and latencies that inform the real world performance results. Parallelism is another issue. Many cores can do work very fast when you're careful about how you split up work between them, taking into consideration the synchronization latency and individual cache sizes and so on. The best approach for doing work on 64 cores can be dramatically different from what works best on 1 or even 2.

That is not a given. A database server can run faster on better hardware because it more effectively and aggressively caches things in memory, which can hurt it on lower-spec systems. Or it can better utilize SIMD instructions that are not present on the low end. Or it is more effective at utilizing more threads, but is slower when run at a low threadcount, etc etc etc.

If I'm not mistaken ClickHouse's min spec is quite steep.

My understanding is that they used c2rust, and then told the LLM/Agents to make the code more idiomatic rust, while also using the PG test suite as a feedback mechanism. This is almost certainly a derived work (and thus a fork, and should thus have the original license and copyright preserved).

For example, if I compiled PG into x86-64 assembly, and then decompiled it into C (via, say, IDA), and then polished that decompiled C code into very readable C code, it is still a derived work. For some reason, people think that if you include an LLM or Agent, copyright can be ignored, and plagiarism is now no longer possible.

It is similar to the crypto-folks thinking that if you use crypto, you no longer have to pay taxes, because the internet/computers make all inconvenient realities go away.

Honestly, such flagrant and arrogant copyright violations make it hard for me to take the project seriously, because it seems like a desperate stunt for attention (which itself may be a solid business move, but that is besides the point).

Put differently, if one were to fork pgrust, strip away the new license and copyright, and restore the original PG license and copyright (while also adding malisper+team to that copyright), they would face no legal consequences at all. In fact, they would probably be a less legal risk than the pgrust team.

There is nothing wrong with wanting to be compensated for your work, but for people like myself which use a cloud managed DB solution (GCP CloudSQL PostgreSQL) it means something like this would never be available.

I consider AGPL a poison pill in my work. That is not true with a suitable commercial license, although I expect a lot more commercial product (support/features/etc). As you note, your objective is to prevent commercialization of your software, but radically speeding up analytics is primarily a concern of large organizations so it seems like a mismatch in purpose.

> If AGPL is an issue for anyone, we would be happy to dual-license under a commercial license.

You should be setting up a CLA process then ASAP if you are looking to take outside contributions.

Sure, that's your prerogative, and kudos for not talking up open source. I'm not amazon size so can't use it, and AGPL is a no go for DB, don't want to be forced to open source my app because I use this!

Will await a MIT based fork myself.

> It’s weird because those who actually care about optimized pg gains are most likely large corporate customers. Why make a product targeting them and license it in such a way they’ll never use it?

Wait, sorry, you're asking why make something enterprise customers might pay for, and then not give it away to them for free?

If it is indeed 300* faster, I'm sure more rational corporations will rethink their license policy or be left in the dust.

AGPL seems like the right choice to me. I’m tired of companies like PlanetScale taking PostgreSQL, building a business on top of it, and then acting like PostgreSQL is theirs to control.

main indeed has two commits, but it clearly states the location of the rest of the commits, so I wouldn't be critical of main itself.

  hey claude, do a breakthrough
  You can find the actual git history at the v0.2 github tag.
  
  Co-Authored-By: Fable <noreply@anthropic.com>

It's easy to claim AI slop nowadays, but you should still mistrust-but-verify.

4K stargazers in a week.

Most official Rust projects are dual MIT/Apache licensed by convention [1] (and most Rust libraries from third parties I've seen that are open source MIT follow the MIT/Apache dual license), so seems like this library shouldn't just be AGPL for a typical political choice of a Rustacean?

[1]: https://rust-lang.org/policies/licenses/

We disabled parallelism in the blog post for demonstration purposes. The 300x slower refers to the clickbench numbers[0] where parallelism is enabled

[0] https://benchmark.clickhouse.com/#system=+liH|pgrs|gQ&type=-...