Hacker News

I think this is a major issue with requiring a PEP for what's a performance optimization.

JITs are big nasty things that by their nature have to do a lot of things. You are basically saying "Give us a PEP for a new python interpreter and compiler".

What they do is fuzzy by their nature because optimizations are very rarely universally beneficial. It's a game of measuring, looking for regressions, and tackling the biggest problem next for whatever sample applications you have available. You always run into the risk of overfitting on top of all that (You may have made django faster, but you've made flask slower).

This is the sort of thing that, IMO, should not go through the PEP process. It's an implementation detail for the runtime.

Imagine, for example, if the JVM needed a bunch of JEPs for JIT improvements. They'd never get done. Or if any javascript engine needed ECMA acceptance before they could do a new JIT optimization.

I am confused. Why can't they have an experimental, optionally-enabled feature for a few years while they iron out all the issues ? What is wrong with that ? The JDK keeps doing this all the while. They write a new JEP if there are changes across previews.

> On the other, it did feel (to me) like with all the fancy (and definitely cool) optimizations it got (automatic refcount operation removal, new type-based specializations, inplace math operations), the macrobenchmark benefits weren't _that_ exciting for a JIT. It definitely has potential for more, maybe with even fancier optimizations like unboxing ints, but the complexity just keeps going up.

Zero chance of shoehorning that into a standards-compliant Python runtime. Some things are too fundamental to change without affecting the user-visible language: JITting, lazy imports, threading, copying gc, ref counting, etc.

I mean, I have had great success with prototyping stuff in python, then letting an llm translate it to C. My 3090 with Gemma4:31b does this very well with a few guidelines.

Dunno how much investment is needed into language optimization anymore.

I think the idea is correct, but hard to retrofit, you sort of have to have it there from the start.

Objective-Smalltalk runs on the Objective-C runtime, so Objective-Smalltalk methods look like Objective-C methods accessed via objc_msgSend() to the entire rest of the system, including to other Objective-Smalltalk code.

That means it doesn't really matter to the rest of the system if any particular method is interpreted, AOT-compiled or JIT-compiled. They can't tell, except for execution speed.

And of course the execution model includes machine primitives, which the interpreter also deals with but doesn't make faster, but the native compilers do.

I like Julia approach, compile everything on first launch, then reuse without speed penality, if you change a function, recompile that.

On another note, I'm working on a (big) PEP for python, and looking for maintainers and sponsors (as per python's rules). Do someone know where I could find such amazing persons? (I already have the code, works in cython,graalVM and pypy, and it's kind of a big change but I believe it will help a lot the language)

This is fine for your own application code, but it doesn't do anything for the libraries you pull in for your project. I don't think that sprinkling Cython code in all library code is going to be a popular solution, especially for those not performance focused. The idea behind a JIT is to be a general performance solution not requiring changes in existing code.

The main reason I don't see it in production code is you significantly increase your chance to write seg faults that kills the runtime / behaves unexpectedly to surrounding systems

I built this in 2001 (https://pyinline.sourceforge.net/). Obviously, that ancient thing doesn't work anymore, but the idea worked then and it ought to work today. That being said, I think there are significantly superior options in existence now!

This is largely the idea behind py2many + fquery, but we want to get rid of C-API as well.

  https://github.com/py2many/py2many
  https://github.com/adsharma/fquery/blob/main/tests/django_example_model.py

This is numba?

We just need Mojo to have 99% Python friendly syntax and we can forget about CPython.

Numba does this. Generally as fast as Cython, but much easier to write.

Numba is basically this.

> How many Python JIT experiments have failed?

Wait, how many? I'd say just this one. The other that I know of, pypy, was massively successful for a single maintainer project. And definitely showed the possibility/potential of JIT python.

I wouldn't call it a failure for not over taking mainline CPython.

The type of dynamic language gymnastics done with Python is huge: any sufficiently well known library basically employs some sort of meta-programming of the language:

  * Typing is a non-obligatory add-on to the syntax
  * SQLAlchemy and other ORMs override basic operators like "=" and employ metaclass programming heavily to create Python-like SQL query builders
  * Pytest rewrites all the assertions as it side-steps bytecode generation (so your .pyc files are different than what they would be without pytest)
  * Many libraries depend on monkey-patching live library calls, including ones to external dependencies
  * The way async was added in is another bolt-on where the same code behaves differently in different contexts
  * Decorators can introduce any side-effect you imagine, and nobody (including core libraries) follow the golden rule of "don't do, only decorate"
  * Many debugging and profiling mechanisms (including REPL) depend on these properties to function — including observability tooling like Sentry, Datadog etc.
  ...

Python is harder to JIT because of the way __dict__ works.

If all python had was __slots__ then we would not be struggling to JIT python.

Tight numerical loops is where the JIT adds performance.

Most performance sensitive features are already available in python between numpy, numba and Cython without really leaving the language core.

You'll mostly be speeding up the slow parts of python, not supercharging the fast parts.

> Maybe base python is more optimized than base Ruby

Not since before Ruby 1.9, no.

I think it would be a mistake to conclude this has anything at all to do with bureaucracy as a matter of inanimate policy that the SC was helplessly in thrall to. Bureaucracy was the tool the SC used to demonstrate that this feature lacked many important properties that they cared about, as mentioned: "maintenance, compatibility, and ... clear success metrics". For whatever detailed reasons, the PEP was withdrawn after it couldn't convincingly demonstrate those things to the satisfaction of the critics on the SC.

Looking at amount of JIT related (for example type confusion) security bugs in V8 I am not surprised other developers are vary of its inclusion.

Large codebases are bureaucracies of their own, that is, the complexity of extra layers starts to bog things down at some point.

In this case, it seems like the human bureaucracy made wise and measured decisions to contain the software bureaucracy. They made reasonable requests to clarify the scope and impact of a major, major core feature. The proposers didn't feel like that was doable, so a complexity nuke was avoided.

The problem with PyPy, which I'm a huge fan of BTW, is it's on life-support. The only reason the past couple major versions were released was due to a few small companies or individuals directly implementing the updates themselves. That's my understanding at least. It's disheartening because I've been primarily using PyPy for 5 years, advocated for it, and was looking into contributing more to it before finding out about its state.

>There are already many way of addressing Python performance (leaving aside whether Python is a good choice if performance matters) such as Cython, Numba, Pypy and C extensions.

And all of them are lacking in different ways, we ALSO, want faster pure Python.

GraalPy and PyPy are great technology achievements, the problem is that the Python community at large rather prefers their C bindings or nowadays Rust, sprinkle a bit of Python FFI on top, and call them "Python" libraries.

> Once your Python warms up

How long does that take?

> It's from converting hashtable lookups into constant-time guard + offset lookups.

Python already does it in a lot of cases, see opcode specializations like LOAD_ATTR_INSTANCE_VALUE, LOAD_ATTR_SLOT.

It still blows my mind that the big "we're breaking backwards compatibility Python 3000" release didn't do anything to lay the groundwork for this, or any of Python's other structural problems.

That team was mostly hired away by NVIDIA.

What's the status of 'Faster CPython'?

I wouldn't be surprised if they all went nodejs instead :/

I mean, if your multi-year-old project is ever asked to just take a moment to write out a quick formal roadmap and justification ... it is doomed, 100% of the time. The actual content of the request doesn't really matter, it's dead.

To be clear, such a request doesn't even mean that someone is consciously attempting to kill the project (although in the corporate world, ime it probably does). It just means that something went critically wrong. In this case, it appears like python's development process really required a PEP for this sort of work, especially since it was happening on the main branch.

CinderX isn't generally faster unless you use "Static Python" which is a major change in how programs are written compared to typical python code.

Getting a JIT from underwhelming to actually good is always going to be a whole lot of iterative work to remove blocking parts of the existing implementation. In Ruby-land we have had a good JIT for a while now, but it took many release cycles and a ton of resources before it was able to outperform the non JITed version when using Rails. Don't expect quick fixes that make great strides and any investments in it needs to be persistent and apply over time.

It boggles my mind that it apparently is the case that such a popular language as Python isn't able to get the resources to do this.

A biased opinion as I dev in F#, but it strikes me that AI is going to play best with functionally pure languages (strongly typed, no side effects, whatever we call that these days)?

F# is on the softest side of that, but letting the compiler tell the AI "nope, here's why" seems like a huge win in saving cycles and time.

The downside at first was simply, "how many examples of F# are there to learn from", and early on it showed, but I think that's been long overcome.

And F# is obviously NOT the best example since it's both "soft" on functional purity and niche. I imagine something like haskell is probably doing even better, although I've yet to check that at all (not that I can code haskell but I haven't looked at anecdotes or anything).

I realize this doesn't really help anyone who's grown to love python, and there's surely no perfect language, but just like how the web catapulted javascript, I see a future where the answer isn't "use AI to make python better" so much as "what already works super well with AI, and does it ease the user adoption"

At this point if you have a python codebase where perf is an issue, the "AI-pilled" answer is to rewrite it in a faster language.

Mature projects simply don't like large unexpected code drops if it is not made clear beforehand what the blast radius is and how it is going to be maintained. On top of that it seems that the performance gains weren't as expected and require further invasive changes.

Implementation effort has never been what has held these efforts back. LLM helps with squirting out code, but the hard tasks are benchmarking, profiling, community engagement, knowledge management, and dealing with the hard questions such as whether the change is actually worth the additional complexity.

These concerns won't go away even if we give up on human developers being able to grok the code and instead completely relying on LLMs for coding.

We pick languages that embrace machine code like Julia, Mojo, Common Lisp, Scheme, and stop calling libraries that are written in C, C++, Fortran, Rust, "Python Libraries", when they are thin bindings.

Clearly the community doesn't want PyPy, GraalPy, or even a CPython JIT.

Python use is widely diverse. Anyone wishing to contribute to that which is published to such a diverse community, all of whom expect Python to continue working for them without any regressions, has to be ready to cater to that diversity in its entirety.

Given that the speedup is modest (~10%) and the Python 3 migration caused enough pain for users, I get it though. If it was a 2x speedup it would be a different story IMO

This is exactly how I write my Python. I'm currently debating between going in deep and transitioning to either TypeScript or Rust. Go's object system and error handling is too weak. Java is too verbose, but I have a soft spot for it.

If you want V8 like optimizations, you have to introduce a strict mode.

Python interpreter is known to leak its implementation (__class_ modification after creation, monkey patching with __getattr__, sys._getframe, eval/exec, deterministic call to __del__, no proper python API for libraries, etc.)

You have to plug all of those holes before having a chance to get a decent JIT. Otherwise it will be just another JIT attempt, like Psyco, Unladen Swallow, Pyston.

It did. It's slower though.

Being slower than a compiled language is not saying much for an interpreted language. But it's slower than PHP which is a closer comparison, and PHP has had JIT for six years now.

It is also slower than any other interpreted dynamic language in wide use.

> strictly typed subset of the language with compilation and static analysis

This is a great idea for popular dynamic languages with existing users and codebases, particularly the focus on "subset" to limit the scope and emphasizing that full backward compatibility is not a goal; and allowing for gradual, incremental adoption. That makes it a lot more practical to implement and adopt.

I think we'll see more of this. Recent examples include strictly typed TypeScript subset designed for ahead-of-time compilation.

That’s effectively what Mojo is. Hardly catching the world on fire but pretty good for what it does.

The performance of python indicates otherwise. There are some performance critical libraries written in C or C++ but a lot of python is written in python.

Python in itself is poorly suited to ever become performant as it lacks the types required. I imagine that causes some issues for effective jit-ing too.

They had a chance to fix this when adding type annotations, but instead of making it "optional/gradual typing" they opted to make it "documentation or linting support", which is obviously the wrong choice, but there were cultural reasons

It looks like it very much should never have happened like this. The actually hard and awkward questions have not been asked beforehand, and the SC doesn't like their hands being forced, and rightly so.

From my limited understanding, the C interop is the biggest obstacle to both JIT and GIL. It's also the feature that makes Python fast enough for so many use cases. And the dynamic typing is the reason it even has to be JIT instead of just pre-compiled.

Unladen Swallow died for lack of upstream interest.

PyPy lacks upstream interest, looked like it could have a shot after many years of development but recently NumPy removed support because they just couldn’t fathom how a non-dead project could be a couple of versions behind CPython.

Now the new JIT is dying because of what seems to be a lack of upstream interest. (The things TFA describes as being required are not unreasonable, but not realistic for a necessarily speculative long-running project without wild amounts of committed funding.)

I’m not optimistic.

> Just think about the sheer amount of electricity and compute power being wasted every single day running inefficient Python across millions of machines worldwide.

Every single time I remember the first AI wave was done with Lisp and Prolog, both with tooling that supports machine code generation since about 50 years.

> committee bureaucracy

That's what's kept the language and ecosystem improving and from becoming a hot mess. If something isn't properly specced then it really shouldn't be in prod. They allowed the experiment in the first place so it'd be easier to gather sufficient data to guide said speccing, but the ball was dropped by the proposers and so the experiment was an ultimate failure (unlike free threading) and prod is now being appropriately cleaned up.

It's not red tape, it's about making sure that the effort actually delivers what it promises and can continue to so in the future. From the state of the draft PEP it seems it doesn't and it won't ever. The SC merely put the nail into the coffin.

I know not everyone loves them; but `Claude rewrite the performance bottlenecks in Rust` works so well it is hard to see the security surface area and harder reason about performance as something that should be paid in core.

cPython is killing it. You are very free to vibe code a new python interpreter that does JIT.

In the remote eventuality that it should actually work and be maintained up to date and secure, I'm sure it will acquire users.

Why is this controversial? We spend hours waiting for python code when it could be much faster if it was treated like a serious programming language

I think it's mainly because of how easy C interop is in Python. Same reason no other interpreter replaced CPython.

While I don't disagree with you in principle, the context for JS JIT vulns and potential Python JIT vulns is pretty different and worth taking into consideration.

For JS, your machine running untrusted JS code is the intended use case. In this scenario the JS is sandboxed in whatever JS engine (V8 etc) and does not have access to "dangerous" methods, and the main use case for a JIT vuln is to escape this sandbox to execute code on the host. This applies to regular internet browsing (and visiting potentially malicious sites) and also, e.g., using the built-in browser in a locked down device (such as game consoles) to gain arbitrary code execution.

For Python, this security boundary doesn't exist. I can't really think of a scenario where you would be running potentially malicious Python code in CPython that would need a JIT vuln to escape whatever sandbox it is in instead of, say, calling the functions in Python directly (or using the C FFI to do whatever). I'm sure that in some niche use cases, this scenario might exist, but to me, it seems like the potential attack surface is fairly minimal.

EDIT: Also, I'm pretty sure that Python's copy-and-patch JIT writes code to a writable mapping and then mprotects it to W^X? I don't think it leaves RWX pages lying around at all.

Who is running sandboxed untrusted Python code?

JS has a completely different threat model. And PHP has just been a basketcase from inception.

> just think about all the exploits in JS engines, as well as the recent PHP vulnerabilities

Can you name some of these JS engine vulnerabilities? And the PHP one had nothing to do with JIT.

This argument would be more convincing if, due to performance reasons, most Python code didn't use C extensions, where exploits are highly prevalent.

I would want a python implementation with a JIT and the ability to choose JIT or no-JIT when defining a python environment. And I want all my libraries to work either way.

I'll get what I want eventually but I might be retired by then.