3.15: gh-142349 backport moves `c_stack_refs` in `_PyThreadStateImpl`, breaking greenlet's rc2-built wheels

[ddorian]

Bug report

Because in RC1 release you said, the following bug is only an issue for wheels built on RC2 and doesn't reproduce on new builds from source:

▎ "There will be no ABI changes from this point forward in the 3.15 series"
▎ "Any binary wheels built against Python 3.15.0 release candidates will work with future versions of Python 3.15."

---

The 3.15 backport of gh-142349 (#158569, 021f634) adds lazy_imports in the middle of _PyThreadStateImpl. On free-threaded builds this moves c_stack_refs from offset 1088 to 1096.

greenlet reads c_stack_refs on every switch. Its cp315t wheels on PyPI were built against rc2, so on the current 3.15 branch they read the wrong field and grow memory until the process is killed. In a pytest run this pushed a 64 GB machine into an OOM in under a minute.

Building greenlet from source fixes it.

Could lazy_imports go at the end of the struct, so wheels built against rc1/rc2 keep working on 3.15.0?

The script below runs one switch in a child process and kills it past 1 GiB:

uv run --python <free-threaded build of the 3.15 branch> greenlet_315_repro.py 

greenlet_315_repro.py

cc @pablogsal @hugovk

CPython versions tested on:

3.15

Operating systems tested on:

Linux

Linked PRs

• gh-158593: Move lazy_imports field to end of _PyThreadStateImpl #158594
• [3.15] GH-158593: Fix ABI compatibility with earlier RCs #158595
• [3.15] gh-158593: Move lazy_imports field to end of _PyThreadStateImpl #158596

[hugovk]

I've opened PR #158594.

[pablogsal]

Wait the thread state is NOT part of the public ABI and therefore not an ABI change.

[hugovk]

It's easy enough to fix, but we've only done so for 3.15, and not main, so greenlet has some time to move away from using these internals.

@ddorian Please open an issue for greenlet. The first 3.16 alpha will be out in a couple of weeks, so you'll be able to start testing against that in GitHub Actions.