CASSPYTHON-3 Introduce pyproject.toml to explicitly declare build dependencies #1264

absurdfarce · 2025-12-23T17:28:28Z

Original motivation is CASSPYTHON-3, a lifted version of PYTHON-1428. These changes also take care of PYTHON-1334 ("Consider migrating dependency, other information in setup.py into pyproject.toml") and PYTHON-1421 ("Adapt setup.py to deprecation of distutils")

This PR represents a pretty significant re-working of setup.py so the question of what kind of release it belongs in is a very real one. The original motivation was to fix a specific problem with the wheels for 3.29.3 but so far I haven't found a way to do that other than the mechanism outlined here. That question is being considered on CASSPYTHON-3; we'll use this PR to discuss and review the changes themselves.

absurdfarce · 2025-12-23T17:30:45Z

Ping @millerjp as well

absurdfarce · 2025-12-23T19:16:45Z

I've run this through our wheel build process for Python 3.12 against ARM64 and x86_64 Ubuntu. In both cases all expected native code was preserved.

Might still have a problem though:

(wheel-validate-venv) mersault@stravinsky:~/Downloads$ python3
Python 3.12.11 (main, Sep  2 2025, 14:20:15) [Clang 20.1.4 ] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from cassandra.cmurmur3 import murmur3
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: /home/mersault/Downloads/wheel-validate-venv/lib/python3.12/site-packages/cassandra/cmurmur3.cpython-312-aarch64-linux-gnu.so: undefined symbol: rotl64
>>> import cassandra.io.libevwrapper as libev
>>>

I mean... at least libev worked...

absurdfarce · 2025-12-23T21:03:01Z

I removed the the "-s" arg to gcc from CFLAGS in order to diagnose the problem:

$ git show HEAD
commit 2d6517477b44cce69faa7c1b9e08b5525c2eb15e (HEAD -> python1428, origin/python1428)
Author: absurdfarce <[email protected]>
Date:   Tue Dec 23 13:21:34 2025 -0600

    Temporarily remove stripping of symbols from shared objects in order to try and diagnose an issue

diff --git a/.github/workflows/build_wheels_linux.yml b/.github/workflows/build_wheels_linux.yml
index c8d5af1..88628ef 100644
--- a/.github/workflows/build_wheels_linux.yml
+++ b/.github/workflows/build_wheels_linux.yml
@@ -13,7 +13,6 @@ jobs:
       CIBW_TEST_REQUIRES: pytest mock pure-sasl eventlet
       CIBW_TEST_COMMAND: echo "wheel is installed"
       CIBW_BEFORE_ALL: yum install -y libev libev-devel
-      CIBW_ENVIRONMENT: CFLAGS='-s'
       CIBW_BUILD_VERBOSITY: 3
     steps:
       - uses: actions/checkout@v4

and the problem... went away?!?!?!?!?!

absurdfarce · 2025-12-23T21:35:40Z

Worth noting that the wheels for 3.29.3 available from pypi also have this issue. Wheels for 3.29.2 from pypi do not have this problem. We introduced the stripping of symbols on 7/30/2024 as the fix for PYTHON-1387. Weird thing is that 3.29.2 was released in September of 2024 so wheels built there would've included that fix.... and they appear fine.

I can't really explain any of this yet. The error itself doesn't make sense... and the chronology doesn't introduce an obvious point where a regression like this could've come in.

absurdfarce · 2025-12-23T22:22:07Z

Moving discussion about the wheel build issue to a distinct issue on python-driver-wheels. Let's keep this PR focused on conversation about the pyproject.toml conversion.

pyproject.toml

absurdfarce · 2026-01-13T00:18:30Z

I've confirmed that these changes (combined with an equivalent set of changes for python-driver-wheels) appear to build valid wheels (including the Cython modules that were missing in CASSPYTHON-3) for all platforms.

Copilot

Pull request overview

This PR migrates the build configuration from a setup.py-centric approach to a modern pyproject.toml-based approach, addressing deprecation of distutils and explicitly declaring build dependencies. The changes significantly refactor setup.py by removing deprecated distutils imports, custom build commands, and pre-build checks while moving project metadata into pyproject.toml.

Changes:

Introduced pyproject.toml with build-system configuration, project metadata, and custom tool configuration for build options
Refactored setup.py to remove distutils usage, simplify extension building logic, and read configuration from pyproject.toml
Removed custom commands (DocCommand), pre-build checks, and complex error handling from setup.py

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 17 comments.

File	Description
pyproject.toml	New file defining build system requirements, project metadata, dependencies, and custom cassandra-driver build configuration options
setup.py	Significantly simplified to focus on extension building, now reads configuration from pyproject.toml and removes deprecated distutils usage

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-13T13:22:24Z

pyproject.toml

@@ -0,0 +1,57 @@
+[build-system]
+build-backend = "setuptools.build_meta"
+requires = ["setuptools", "Cython>=3.0", "toml", "cassandra-driver"]


Critical circular dependency issue: The build-system requires "cassandra-driver" as a build dependency, but this is the package being built. This will cause build failures as pip cannot install cassandra-driver before building cassandra-driver. The "cassandra-driver" entry should be removed from the requires list.

Suggested change

requires = ["setuptools", "Cython>=3.0", "toml", "cassandra-driver"]

requires = ["setuptools", "Cython>=3.0", "toml"]

Copilot · 2026-01-13T13:22:25Z

setup.py

-class build_extensions(build_ext):
-
-    error_message = """
+windows_compmile_error = \


Spelling error: "compmile" should be "compile".

Suggested change

windows_compmile_error = \

windows_compile_error = \

Copilot · 2026-01-13T13:22:25Z

setup.py

+try_murmur3 = driver_project_data["build-murmur3-extension"]
+try_libev = driver_project_data["build-libev-extension"]
+try_cython = driver_project_data["build-cython-extensions"] and not is_pypy



The refactored code removes support for command-line arguments that were present in the original: --no-murmur3, --no-libev, --no-cython, and --no-extensions. This is a breaking change that could affect users and build scripts that relied on these flags to control extension building. Consider either maintaining backward compatibility or documenting this breaking change clearly.

Suggested change

# Backwards-compatible support for legacy command-line flags that used to

# control extension building. These flags override the configuration in

# pyproject.toml and are removed from sys.argv so that setuptools and other

# tooling do not see them.

legacy_flag_map = {

"--no-murmur3": "murmur3",

"--no-libev": "libev",

"--no-cython": "cython",

"--no-extensions": "all",

}

cleaned_argv = []

for _arg in sys.argv:

flag_type = legacy_flag_map.get(_arg)

if flag_type is None:

cleaned_argv.append(_arg)

continue

if flag_type == "murmur3":

try_murmur3 = False

elif flag_type == "libev":

try_libev = False

elif flag_type == "cython":

try_cython = False

elif flag_type == "all":

try_murmur3 = False

try_libev = False

try_cython = False

sys.argv = cleaned_argv

Copilot · 2026-01-13T13:22:25Z

setup.py

+setup(
+    version = __version__,
+    ext_modules = build_extension_list()
+)


The refactored code removes the check that warned users if the old standalone cqlengine package was installed alongside the driver. This warning helped prevent conflicts since cqlengine is now integrated into the driver. Removing this check could lead to confusing behavior for users who have both packages installed. Consider whether this warning is still needed or if it can be safely removed.

Copilot · 2026-01-13T13:22:26Z

pyproject.toml

@@ -0,0 +1,57 @@
+[build-system]
+build-backend = "setuptools.build_meta"
+requires = ["setuptools", "Cython>=3.0", "toml", "cassandra-driver"]


The build-system requires "Cython>=3.0" unconditionally, but in the original code, Cython was only added to setup_requires when actually needed (when try_cython was True and pre_build_check passed). This means Cython will now always be installed during the build process, even if the user doesn't want to build Cython extensions. This could increase build time and dependencies unnecessarily. Consider making Cython an optional build dependency or documenting why it's now always required.

Copilot · 2026-01-13T13:22:28Z

setup.py

+setup(
+    version = __version__,
+    ext_modules = build_extension_list()
+)


The refactored code removes the DocCommand custom command that was used to generate and test documentation via python setup.py doc or python setup.py doc --test. This is a breaking change that could affect documentation build workflows. Consider either maintaining this functionality or documenting the migration path for documentation building.

Copilot · 2026-01-13T13:22:28Z

setup.py

-    has_subprocess = False
+from pathlib import Path
+from setuptools import setup, Extension
+import toml


The code imports and uses the toml library, which is also listed as a build dependency. However, for Python 3.11+, the standard library includes tomllib for reading TOML files. For Python 3.9-3.10, the recommended approach is to use tomli (the backport of tomllib) rather than the older toml package. Consider using tomllib (with tomli as a fallback for older Python versions) for better compatibility and future-proofing.

Suggested change

import toml

try:

import tomllib as toml

except ImportError:

import tomli as toml

Copilot · 2026-01-13T13:22:28Z

setup.py

+        except Exception:
+            sys.stderr.write("Failed to cythonize one or more modules. These will not be compiled as extensions (optional).\n")


The exception handler at line 172 catches all exceptions with a bare except Exception:, which will silently suppress all errors during Cythonization. While this matches the original behavior, it would be better to log the actual exception details (e.g., using sys.stderr.write(str(exc))) to help users diagnose build issues rather than just printing a generic message.

Suggested change

except Exception:

sys.stderr.write("Failed to cythonize one or more modules. These will not be compiled as extensions (optional).\n")

except Exception as exc:

sys.stderr.write("Failed to cythonize one or more modules. These will not be compiled as extensions (optional).\n")

sys.stderr.write("Cythonization error: %s\n" % (exc,))

Copilot · 2026-01-13T13:22:29Z

setup.py

+try_murmur3 = driver_project_data["build-murmur3-extension"]
+try_libev = driver_project_data["build-libev-extension"]
+try_cython = driver_project_data["build-cython-extensions"] and not is_pypy


The logic for try_murmur3 and try_libev does not respect the try_extensions flag. These extensions could be built even when try_extensions is False (e.g., on unsupported platforms). These variables should be combined with try_extensions similar to how it was done in the original code. For example: try_murmur3 = try_extensions and driver_project_data["build-murmur3-extension"].

Suggested change

try_murmur3 = driver_project_data["build-murmur3-extension"]

try_libev = driver_project_data["build-libev-extension"]

try_cython = driver_project_data["build-cython-extensions"] and not is_pypy

try_murmur3 = try_extensions and driver_project_data["build-murmur3-extension"]

try_libev = try_extensions and driver_project_data["build-libev-extension"]

try_cython = try_extensions and driver_project_data["build-cython-extensions"] and not is_pypy

Copilot · 2026-01-13T13:22:29Z

pyproject.toml

+    "Programming Language :: Python :: Implementation :: CPython",
+    "Programming Language :: Python :: Implementation :: PyPy",
+    "Topic :: Software Development :: Libraries :: Python Modules"
+]


The pyproject.toml is missing a requires-python field to specify the minimum Python version required. While the classifiers list Python 3.9-3.13, the requires-python field should be explicitly set (e.g., requires-python = ">=3.9") to ensure pip enforces this requirement during installation.

Suggested change

]

]

requires-python = ">=3.9"

absurdfarce added 3 commits December 22, 2025 15:51

Radically simplified setup.py plus the addition of pyproject.toml

51dc632

Move env var configs into settings within pyproject.toml

8b21e54

Forgot to remove local paths for libev install

fcfc680

absurdfarce marked this pull request as draft December 23, 2025 17:28

absurdfarce requested a review from bschoening December 23, 2025 17:29

bschoening requested changes Dec 24, 2025

View reviewed changes

pyproject.toml Show resolved Hide resolved

pyproject.toml Show resolved Hide resolved

Change a few names around

475df4b

absurdfarce marked this pull request as ready for review January 13, 2026 00:01

Review feedback

f30f0a8

absurdfarce requested a review from bschoening January 13, 2026 00:08

absurdfarce mentioned this pull request Jan 13, 2026

Wheel build changes necessary to support CASSPYTHON-3 datastax/python-driver-wheels#17

Open

bschoening requested a review from Copilot January 13, 2026 13:16

Copilot started reviewing on behalf of bschoening January 13, 2026 13:16 View session

bschoening approved these changes Jan 13, 2026

View reviewed changes

Copilot AI reviewed Jan 13, 2026

View reviewed changes

	requires = ["setuptools", "Cython>=3.0", "toml", "cassandra-driver"]
	requires = ["setuptools", "Cython>=3.0", "toml"]

+# Backwards-compatible support for legacy command-line flags that used to
+# control extension building. These flags override the configuration in
+# pyproject.toml and are removed from sys.argv so that setuptools and other
+# tooling do not see them.
+legacy_flag_map = {
+    "--no-murmur3": "murmur3",
+    "--no-libev": "libev",
+    "--no-cython": "cython",
+    "--no-extensions": "all",
+}
+cleaned_argv = []
+for _arg in sys.argv:
+    flag_type = legacy_flag_map.get(_arg)
+    if flag_type is None:
+        cleaned_argv.append(_arg)
+        continue
+    if flag_type == "murmur3":
+        try_murmur3 = False
+    elif flag_type == "libev":
+        try_libev = False
+    elif flag_type == "cython":
+        try_cython = False
+    elif flag_type == "all":
+        try_murmur3 = False
+        try_libev = False
+        try_cython = False
+sys.argv = cleaned_argv

-import toml
+try:
+    import tomllib as toml
+except ImportError:
+    import tomli as toml

		except Exception:
		sys.stderr.write("Failed to cythonize one or more modules. These will not be compiled as extensions (optional).\n")

CASSPYTHON-3 Introduce pyproject.toml to explicitly declare build dependencies #1264

Are you sure you want to change the base?

CASSPYTHON-3 Introduce pyproject.toml to explicitly declare build dependencies #1264

Uh oh!

Conversation

absurdfarce commented Dec 23, 2025

Uh oh!

absurdfarce commented Dec 23, 2025

Uh oh!

absurdfarce commented Dec 23, 2025

Uh oh!

absurdfarce commented Dec 23, 2025

Uh oh!

absurdfarce commented Dec 23, 2025

Uh oh!

absurdfarce commented Dec 23, 2025

Uh oh!

Uh oh!

Uh oh!

absurdfarce commented Jan 13, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants