Move from setup.py to uv+pyproject.toml

[birdcar]

Description

I ran this by @nicknisi, but this is an upgrade to the plumbing behind our package building a testing to use UV instead of setup.py.

"Why?" I hear you asking; ultimately a few reasons.

• Speed of development: UV is fast as hell and can manage/install its own versions of python, which means testing against multiple versions of Python is easier / more straightforward (I will open some future PRs to make this a single command, in fact)
• Keeping up with Python conventions: There are still a lot of tutorials and old recommendations laying around that call for a setup.py file and direct invocation, but all direct invocations of setup.py have been essentially deprecated for years, and since PEP 518/621 landed, it's preferred to move away from the file entirely towards non-code files (namely, pyproject.toml).
• Reproducibility: Moving to a src/ layout (i.e. the python package is in the src directory instead of the root directory) and managing its installation with uv also means we avoid some common and easy to hit bugs when it comes to catching import problems etc.

There are other reasons I could probably elucidate, but I genuinely think this is a fairly uncontroversial change since a lot of the Python community is running to use UV as it stands.

Importantly, this does not change user workflows at all, with the minor exception of affecting those who were executing python setup.py install directly previously (I'm unclear if there's a way to know that up front. We can bump the version and communicate this as a breaking change if so).

Documentation

Does this require changes to the WorkOS Docs? E.g. the API Reference or code snippets need updates.

[ ] Yes

If yes, link a related docs PR and add a docs maintainer as a reviewer. Their approval is required.

[greptile-apps]

Skipped: This PR changes more files than the configured file change limit: (150 files found, 100 file limit)

[birdcar]

Just realized I need to update the release workflow as well. Don't merge before I do that (should only take a moment or two)

[birdcar]

The release workflow has been updated now, and according to the UV docs the way we should go about authentication for publishing a package is to configure this release action as a trusted publisher on PyPI.

I can only make part of that process happen, namely, I can follow the guide for creating the release workflow, including the recommended environment (which I did in .github/workflows/release.yml). The rest will, I'm sure, need to be discussed or acted on by someone else (@nicknisi maybe?).

[gjtorikian]

Given that

• uv is the absolute best; and
• ~80% of this PR is just files being moved around

I'm very happy to merge these changes.

However, the one thing that makes me nervous are the changes to the workflows. IIRC, GitHub isn't running the ci.yml in this branch; and, we won't know if the release.yml is working correctly until there's an actual release. In other words, I'd hate to merge this in, only to discover that there's more work pending for those automations.

wdyt, @birdcar ? After this is merged, is there another branch that could be updated with main to test CI and the release flow? (As noted, for example, the environment should be created before the merge. I think all of that release testing should be done in sync asap just to make sure there are not surprises.)

[birdcar]

@gjtorikian I can absolutely split those out. I think my only concern there is that the current workflows will for sure be broken after merge with this update. Having said that:

IIRC, GitHub isn't running the ci.yml in this branch

Afaict GitHub is in fact running the updated CI workflows on this branch (note the "UV" specific pieces in the completed checks).

The only workflow that should need additional TLC is the release workflow. I'm happy to split just that workflow out if we'd like or to merge it and then push a patch release to debug.

I think my only real concern is that merging this will immediately break all existing PRs and require them to rebase. If that's acceptable to folks for the gains then that's great, if not we can focus on getting those merged or closed and then merge this in after.

EDIT: added an additional concern.

[gjtorikian]

Afaict GitHub is in fact running the updated CI workflows on this branch

Derp, I didn't even check the run. Sorry about that! I think back in the day workflows didn't run in the branches they were changed in.

I'm happy to split just that workflow out if we'd like or to merge it and then push a patch release to debug.

For me, it's less about spinning it out and more about the latter: pushing a release and watching it do its thing, setting aside time to debug it if necessary. Just to move this along, how about splitting out the release.yml change, then working with $someone to a) make the PyPi change, b) make the GitHub environment change, and c) cut the release?

I think my only real concern is that merging this will immediately break all existing PRs and require them to rebase.

I'm in the process of updating/merging existing PRs. One is a draft, one is from a coworker, and another was opened a couple days ago. If #430 is responded to, I can merge it in and that'll officially be good enough to justify a minor version.

I think the pain of a rebase is well worth the DX improvement of uv and I volunteer as tribute to get everyone's branches nice and tidy. 🫡

[birdcar]

Just to move this along, how about splitting out the release.yml change, then working with $someone to a) make the PyPi change, b) make the GitHub environment change, and c) cut the release?

Can do. I'm out for the day so this will be a tomorrow priority ✌🏽