Hacker News

Some overlap here with some of my favorite essays on why SQL is lacking:

https://www.scattered-thoughts.net/writing/against-sql

That particular post ends with a wish-list of items so it's the most similar to the OP. But there are others on the site that I quite enjoy (click on the home icon and search "SQL" on the page).

My personal take is that SQL will continue to reign for a long time because of the how monumental the task of replacing it is due to the inherent complexity of databases. LLMs make this worse because they're really good at translating prose to SQL. Now that it matters less how annoying SQL is to programmers, SQL will become more like assembly over time: something mostly computers write because it's complicated for humans to deal with directly. This is deeply ironic given that SQL was ostensibly designed to read like prose, i.e. to be easy for humans.

I would be genuinely curious what the OP other and folks here think of Mangle Datalog in this context. The go implementation is here: https://codeberg.org/TauCeti/mangle-go and the Rust implementation is here: https://codeberg.org/TauCeti/mangle-rs

The implementations are not high performance, but if you can fit everything in memory or you can organize your data and integrate it through external queries, you should get something workable for a lot of use cases.

I did not set out to replace SQL, and while I don't mind adoption, that is not why I am sharing it here. The open sourcing was motivated by making datalog more widely known. I did some research and found out that I needed a datalog implementation with particular characteristics, I for sure knew I didn't want to use SQL for what I needed.

There are structured types and recursion and being able to name predicates and compose queries... Mangle has some users and there is a few application that take advantage of the queries-as-logic-programming approach.

I think an insight one can draw in this discussion that a query language and the system (DBMS implementation) that it is part of can hardly be separated when it comes to the inevitable performance requirements one has.

I've tend to use both SQL and a dataframe API library. I guess it is sort of a a best of both worlds. SQL for the high level, large aggregations, dataframe library for when the equivalent SQL statement would be very tedious or hard to debug.

As a meta comment, I can handle code blocks without syntax highlighting, and I can handle code blocks that wrap. But both together with long comments just turn into line noise. There's no longer any useful visual signal for how to read them. On my phone the code blocks are simply impossible to meaningfully parse.

https://www.geldata.com/blog/we-can-do-better-than-sql

(https://news.ycombinator.com/item?id=24106608, https://news.ycombinator.com/item?id=19871051)

Sounds a lot like Spark before it became so enterprise-focused. Back in my day we wrote scala to run our queries, and once we figured out how to get our compiler and runtime set up, we liked it!

I’ve been getting into Postgres recently and I was very surprised how easy it is to introduce new types/operators/etc through C code. I’m not talking about domains. Just write some C and you can have whatever type you want. It really demystified “extensions” for me, I actually think that is an actively harmful name (it sounds clunky, gross, based on my experience dealing with “extension” and “plugins” elsewhere) for what is essentially just custom types/functions. More people should try writing their own postgres extensions. It’s not very difficult at all!

I’ve been cooking in this space for quite a while (HDFS/spark, Apache Pinot, proprietary stuff, an experimental functional ORM over SQLite). The biggest problem, I think, is the interface between the management/admin, application, and “query” layers. I think something like grpc/protoc (or indeed the way Spark used the JVM) is needed to provide non-leaky abstractions and more programmatic/structured interfaces from the DB to its clients. Happy to share more, but basically, the database needs to become capable of general (meta-)parsing with a reflective type system, I think.

My ask is 15 years old[1], a live SQL extension. Allow a query to be a subscription to a database, so any updates get streamed as deltas to a listening client. There were a ton of times in my time using SQL where the same query is run over and over, just to get/handle that delta.

Wouldn't it be a lot more efficient to just work that way in the first place?

[1] http://livesql.org/ <--- just a few paragraphs of text from 2011

PRQL is one of the best attempts at a new query language IMO

I've been working on a Lean4-based query lang that compiles to substrait, I think the power it has wrt to types and functional programming could improve on SQL ergonomics a good deal

Can someone explain to me why SQL error messages are so bad? I routinely have some monster query where the message is effectively, "Illegal syntax somewhere, dufus".

Datalevin seems to fit the bill https://datalevin.org/docs

First paragraph of preface: “SQL has been the default language of application databases for half a century. That default is now holding application state back. Datalevin is a database built to replace SQL databases at the center of application systems: it stores data as small facts and queries those facts with Datalog”

SQL is what it is today because it is battle tested and has to handles a very hard problem of handling arbitrary concurrent reads/writes, so the likely scenario is that trying to replace general SQL wholesale will just end up making a worse, less tested version of SQL that developers are less familiar with. So, I think the best query language is probably whatever query feature that's already in your backend language, LINQ for C# for example. The only room for an SQL replacement in my opinion is if you are willing to trade flexibility for speed a la TigerBeetle.

The good thing about having built your own programming language via LLM nowadays is that you don't really have to speculate about a theoretical language when you can just have Codex/Claude implement it and try it out for yourself. I did it yesterday when I wanted to try out this theoretical high-performance database architecture that I had in mind and just added query functionalities to the language I already have.

If anyone is interested about the results, the default naive mode for this new database is ~0.2x the speed of concurrent durable mutation workloads, but if you specialize it to the particular application, you can get ridiculous 50-100x performance increases on filters and maps at the cost of flexibility and more upfront design. Experimental results are promising, definitely not production ready though.

The problem with alternative query languages is that the people who have the most knowledge about creating queries and of the relational domains underlying their businesses are all experts in SQL. Introducing something else, then, means your most natural user base must migrate away from something they understand how to use well, and that's a hard sell.

So, until the ultimate query language is developed, I'll take SQL with pipes. It's an easy sell and good enough to eliminate 90% of my gripes about SQL.

this is exactly why i built Rad: a relational db with an IR as its public interface, so you can experiment with interesting query languages against a solid foundation and a real planner without needing to compile-up to sql (radengine.dev)

Nothing around better error handling or schema updates? After working with SQL for 10+ years those are definitely the things I miss.

For error handling I mean things like deprecating a column and allowing a custom error message when someone queries it.

And for schema updates I mean allowing table versions. Same table name but allowing querying an older version of the schema

SQL is the worst way to access a database, except for all of the others that have been tried from time to time

I don't want a query language. I want to call and profile typed functions like normal data structures, and which use (low-level/non-declarative) RPC where needed

This would have been interesting about a decade ago, but today AIs all know SQL, and I haven't written it myself in a while.

Since it seems like the quantity of training data dominates AI performance, and AI doesn't yet internalize experience with new tools, it seems like a bad idea to stray from the training set.

Without repeatable benchmarks, it feels like obsessing over a language's syntax and semantics feels a little like debating whether you write assembly using AT&T or Intel syntax.

for most 'application' like workloads not analytics - standardizing or improving on mongo-query language (MQL) would be welcome.

the drawback is your query patterns have to be known before hand when designing your application. which isn't really a drawback since you're doing it before building the application. & hence not as flexible as SQL.

I think part of the issue is that SQL is nice for some things (do some aggregation on a row-filtered subset of columns) but perhaps not as much for other things (a query where later rows depend on earlier rows in complex ways). I think being able to compile a procedural programming language to SQL would be pretty nice for the latter.

Check out CodeQL, it's a modern relational query language based on Datalog.

This is an ultra compact relational query language designed for LLMs. https://memelang.net/11/

I wan't easier ways to work with nested structures, like relationships. All this flat table structure is a pain.

Database engineers have to be the change they want to see and add support for newer query languages.

Why don't we have APL as the language here?