1791980 – mingw-python3 fails to build with Python 3.9: /bin/sh: python: command not found

Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.

Bug 1791980 - mingw-python3 fails to build with Python 3.9: /bin/sh: python: command not found

Summary: mingw-python3 fails to build with Python 3.9: /bin/sh: python: command not found

Keywords:
Status:	CLOSED RAWHIDE
Alias:	None
Product:	Fedora
Classification:	Fedora
Component:	mingw-python3
Sub Component:
Version:	rawhide
Hardware:	Unspecified
OS:	Unspecified
Priority:	unspecified
Severity:	unspecified
Target Milestone:	---
Assignee:	Sandro Mani
QA Contact:	Fedora Extras Quality Assurance
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:	PYTHON39
TreeView+	depends on / blocked

Reported:	2020-01-16 19:22 UTC by Miro Hrončok
Modified:	2020-05-28 21:07 UTC (History)
CC List:	4 users (show)
Fixed In Version:
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:	2020-05-28 21:07:44 UTC
Type:	Bug
Embargoed:
Dependent Products:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Python	40777	0	None	None	None	2020-05-26 09:56:30 UTC

Description Miro Hrončok 2020-01-16 19:22:20 UTC

mingw-python3 fails to build with Python 3.9.0a2:

fi
/usr/lib/gcc/i686-w64-mingw32/9.2.1/../../../../i686-w64-mingw32/bin/ld: .rsrc merge failure: duplicate leaf: type: 18 (MANIFEST) name: 2 lang: 409
/usr/lib/gcc/i686-w64-mingw32/9.2.1/../../../../i686-w64-mingw32/bin/ld: .rsrc merge failure: duplicate leaf: type: 10 (VERSION) name: 1 lang: 409
i686-w64-mingw32-gcc      -municode -o python.exe Programs/python.o -L. -lpython3.8 -lpthread -lm -lversion -lshlwapi -lws2_32            -lm  python_exe.o
i686-w64-mingw32-gcc      -municode -mwindows -o pythonw.exe Programs/python.o -L. -lpython3.8 -lpthread -lm -lversion -lshlwapi -lws2_32            -lm   pythonw_exe.o
i686-w64-mingw32-gcc      -o Programs/_testembed Programs/_testembed.o -L. -lpython3.8 -lpthread -lm -lversion -lshlwapi -lws2_32            -lm 
_PYTHON_PROJECT_BASE=/builddir/build/BUILD/Python-3.8.1/build_win32 _PYTHON_HOST_PLATFORM=mingw PYTHONPATH=../Lib _PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata__win_ python -S -m sysconfig --generate-posix-vars ;\
if test $? -ne 0 ; then \
	echo "generate-posix-vars failed" ; \
	rm -f ./pybuilddir.txt ; \
	exit 1 ; \
fi
/bin/sh: python: command not found
generate-posix-vars failed


Does mingw-python3 need to be updated together with python3?


For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.9/fedora-rawhide-x86_64/01148588-mingw-python3/

For all our attempts to build mingw-python3 with Python 3.9, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.9/package/mingw-python3/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.9:
https://copr.fedorainfracloud.org/coprs/g/python/python3.9/

Let us know here if you have any questions.

Python 3.9 will be included in Fedora 33. To make that update smoother, we're building Fedora packages with early pre-releases of Python 3.9.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Sandro Mani 2020-01-17 16:39:15 UTC

I'm looking into it.

> Does mingw-python3 need to be updated together with python3?

Yes, since mingw{32,64}-python3 is a shell script which ends up invoking the native python, i.e.

#!/usr/bin/sh
pylibdynload=$(/usr/bin/python3.8 -c 'import sysconfig; import os; print(os.path.join(sysconfig.get_path("platstdlib"), "lib-dynload"))')
CC=x86_64-w64-mingw32-gcc LD=x86_64-w64-mingw32-ld DLLWRAP=x86_64-w64-mingw32-dllwrap _PYTHON_HOST_PLATFORM=mingw _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata__win_" PYTHONHOME=/usr/x86_64-w64-mingw32/sys-root/mingw PYTHONPATH=$PYTHONPATH:/usr/x86_64-w64-mingw32/lib/python3.8:/usr/x86_64-w64-mingw32/lib/python3.8/site-packages:$pylibdynload:/usr/x86_64-w64-mingw32/sys-root/mingw/lib/python3.8:/usr/x86_64-w64-mingw32/sys-root/mingw/lib/python3.8/site-packages /usr/bin/python3 "$@"

Comment 2 Miro Hrončok 2020-01-17 16:49:59 UTC

OK. I'll let you know here once we push 3.9 to Fedora proper.

Comment 3 Sandro Mani 2020-01-17 18:03:51 UTC

So the source of this particular build failure is [1], host python version needs to match the version being cross-compiled.

Here is some initial work to build mingw-python3-3.9 [2]. I can finish this as soon as 3.9 is in Fedora proper.

[1] https://github.com/python/cpython/blob/master/configure.ac#L67
[2] https://src.fedoraproject.org/rpms/mingw-python3/tree/python3.9

Comment 4 Ben Cotton 2020-02-11 17:33:43 UTC

This bug appears to have been reported against 'rawhide' during the Fedora 32 development cycle.
Changing version to 32.

Comment 5 Miro Hrončok 2020-05-25 13:01:21 UTC

This comment is mass posted to all bugs blocking the Python 3.9 tracker, sorry if it is not 100 % relevant. When in doubt, please ask.


The Python 3.9 rebuild is in progress in a Koji side tag.

If you fix this bug, please don't rebuild the package in regular rawhide, but do it in the side tag with:

    $ fedpkg build --target=f33-python

The rebuild is progressing slowly and it is possible this package won't have all the required build dependencies yet. If that's the case, please just leave the fix committed and pushed and we will eventually rebuild it for you.

You are not asked to go and try rebuild all the missing dependencies yourself. If you know there is a bootstrap loop in the dependencies, let me know and we can untangle it together.

If you want to test your fix or reproduce the failure, you can still use the Copr repo mentioned in the initial comment of this bug: https://copr.fedorainfracloud.org/coprs/g/python/python3.9/

Comment 6 Sandro Mani 2020-05-26 09:20:16 UTC

I'm hitting this

/builddir/build/BUILD/Python-3.9.0b1/Modules/_datetimemodule.c:3328:16: error: initializer element is not constant
 3328 |     .tp_base = &PyTuple_Type,

with mingw-python3 but you don't seem to be hitting this with the native build? The error looks correct, as PyTuple_Type does not have static storage. Does any of this ring any bell?

Comment 7 Petr Viktorin 2020-05-26 09:30:36 UTC

A faint bell; as far as I remember, things like this tend to only break on Windows when building extensions as dynamic libraries (rather than built into the main executable, which the upstream MSVC builds do AFAIK).
Please open an upstream bug.

Comment 8 Miro Hrončok 2020-05-26 10:05:38 UTC

BTW sorry for pushing the "Rebuilt for Python 3.9" commit and the build attempt. I forgot to blacklist mingw-python3.

Comment 9 Petr Viktorin 2020-05-28 16:04:30 UTC

I sent a pull request to fix this issue: https://github.com/python/cpython/pull/20493

If you have time to test the patch with mingw, maybe that could uncover more issues.
If the patch is merged upstream, I plan to apply it to the python3.9 package after the side tag is merged.

Comment 10 Sandro Mani 2020-05-28 21:07:44 UTC

Thanks, mingw-python3-3.9.0-0.1.b1.fc33 is built in f33-python with this patch.

Note You need to log in before you can comment on or make changes to this bug.