How to build an sdist and then build and install a wheel from the sdist?

[wshanks]

I would like for tox to build an sdist and then build and install a wheel from that. From what I can tell tox's default behavior is to build either an sdist or wheel but not both.

I tried using this configuration:

[testenv]
package = wheel
wheel_build_env = .pkg
commands = python -m testtox
setenv =
  PYTHONSAFEPATH=1

[testenv:.pkg]
deps = build
commands = python -m build

but I saw that specifying deps in the build environment should not be supported according to #3412 (comment) so this can't be right.

I made a repo here with my experiment. As the readme says, tox run prints out hello, tox even though MANIFEST.in should exclude __main__.py while python -m build does exclude __main__.py from the build artifacts.

So the main question is -- how can I build an sdist and then build a wheel from the sdist? There might be a secondary issue with tox's wheel not respecting the manifest.

One bonus request -- I was hoping I could leave package = editable as the default only go down this sdist-wheel path when using --override testenv.package=wheel.

[wshanks]

I found a way to get python -m build working by using package = external. I actually found this in my notes from a while ago so maybe I got this from somewhere in a previous issue/discussion on the repo but I couldn't find it when searching again recently 😞.

[testenv]
package = external
package_env = .pkg
wheel_build_env = .pkg
commands = python -m testtox
setenv =
  PYTHONSAFEPATH=1

[testenv:.pkg]
deps = build
package_glob = {envtmpdir}/dist/*.whl
commands = python -m build -o {envtmpdir}/dist

[jugmac00]

I don not think this is a supported use case currently. Could you elaborate on your use case for this scenario?

[wshanks]

The goal is to test with the wheel that is generated from the sdist because build backends like setuptools and hatchling allow for what is in the sdist and wheel to be customized and so what gets packaged in the sdist might not be sufficient to build the wheel. There is some discussion of this topic in this discuss.python.org thread and the issue in the pypa/build repo linked in the first post. I used to achieve this behavior in tox 3 with tox-wheel (ionelmc/tox-wheel#13).

I think you are saying that tox does not directly support building the wheel from the sdist. Does tox support the package = external solution I gave in my second post? I could not find much information on wheel_build_env and package = external besides the suggestion to set wheel_build_env = .pkg for projects that can build the wheel for all Python versions from a single environment. The suggested configuration seems to work for me. The one problem I encountered was having different environments with package = editable and package = external with wheel_build_env set like this. I needed to set wheel_build_env to separate environments for the different package types.

Also, in practice, what I have now is package = editable in tox.ini like this (same as above except the second line):

[testenv]
package = editable
package_env = .pkg
wheel_build_env = .pkg
commands = python -m testtox
setenv =
  PYTHONSAFEPATH=1

[testenv:.pkg]
deps = build
package_glob = {envtmpdir}/dist/*.whl
commands = python -m build -o {envtmpdir}/dist

In this case, an editable install is used by default but when I use tox --override testenv.package=wheel run -epy tox instead builds the wheel from the sdist. I worry a little that this might not be supported because the configuration in .pkg is ignored by default which feels weird, but this works for me for now at least.

[wshanks]

I found one other problem with my suggested configuration above. It seems that variable substitution is not performed in the wheel_build_env, so the packaging outputs end up in a directory literally named {envtmpdir} instead of in .tox/py/tmp or .tox/.pkg/tmp.

[gaborbernat]

Your package = external solution is the right approach for this use case. For others finding this discussion, the working config is:

[testenv]
package = external
package_env = .pkg

[testenv:.pkg]
deps = build
package_glob = {envtmpdir}/dist/*.whl
commands = python -m build -o {envtmpdir}/dist

This builds an sdist first (default python -m build behavior), then builds a wheel from that sdist, and tox picks up the wheel via package_glob for installation into the test env.