Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1897112 - libvirt-python fails to build with Python 3.10: setup.py test truncates Python version to 3.1
Summary: libvirt-python fails to build with Python 3.10: setup.py test truncates Pytho...
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: libvirt-python
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Daniel Berrangé
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: PYTHON3.10
TreeView+ depends on / blocked
 
Reported: 2020-11-12 10:46 UTC by Tomáš Hrnčiar
Modified: 2021-02-02 16:53 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-02-02 16:53:40 UTC
Type: Bug
Embargoed:

Attachments (Terms of Use)

Description Tomáš Hrnčiar 2020-11-12 10:46:53 UTC 
libvirt-python fails to build with Python 3.10.0a2.

Traceback (most recent call last):
  File "/builddir/build/BUILD/libvirt-python-6.9.0/sanitytest.py", line 5, in <module>
    import lxml.etree
ImportError: /usr/lib64/python3.10/site-packages/lxml/etree.cpython-310-x86_64-linux-gnu.so: undefined symbol: _PyGen_Send
 
This seem to be related to https://github.com/cython/cython/issues/3876. 
It might be fixed with this patch https://github.com/cython/cython/pull/3877 that works for numpy. Upstream has decided to solve this differently, but until that happens, we are using this patch in our Copr (for other packages).

For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.10/fedora-rawhide-x86_64/01757137-libvirt-python/

For all our attempts to build libvirt-python with Python 3.10, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.10/package/libvirt-python/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.10:
https://copr.fedorainfracloud.org/coprs/g/python/python3.10/

Let us know here if you have any questions.

Python 3.10 will be included in Fedora 35. To make that update smoother, we're building Fedora packages with early pre-releases of Python 3.10.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.
Comment 1 Daniel Berrangé 2020-11-12 11:10:39 UTC 
That stack trace shows the problem is in the lxml.etree  module, not libvirt, so re-assigning.
Comment 2 Tomáš Hrnčiar 2020-11-12 11:18:24 UTC 
Yes, sorry for the confusion. There was a mid air collision while I was updating the bugzilla. 

The above error was fixed in libxml, but libvirt-python rebuild leads to another error:

libvirt-lxc-override.c:67:5: warning: ‘PyEval_ThreadsInitialized’ is deprecated [-Wdeprecated-declarations]
   67 |     LIBVIRT_BEGIN_ALLOW_THREADS;
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/include/python3.10/Python.h:146,
                 from libvirt-lxc-override.c:17:
/usr/include/python3.10/ceval.h:121:36: note: declared here
  121 | Py_DEPRECATED(3.9) PyAPI_FUNC(int) PyEval_ThreadsInitialized(void);
      |                                    ^~~~~~~~~~~~~~~~~~~~~~~~~
libvirt-lxc-override.c:69:5: warning: ‘PyEval_ThreadsInitialized’ is deprecated [-Wdeprecated-declarations]
   69 |     LIBVIRT_END_ALLOW_THREADS;
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/include/python3.10/Python.h:146,
                 from libvirt-lxc-override.c:17:
/usr/include/python3.10/ceval.h:121:36: note: declared here
  121 | Py_DEPRECATED(3.9) PyAPI_FUNC(int) PyEval_ThreadsInitialized(void);

You can see full log here: https://download.copr.fedorainfracloud.org/results/@python/python3.10/fedora-rawhide-x86_64/01766717-libvirt-python/builder-live.log.gz

https://docs.python.org/3.10/whatsnew/changelog.html#id48
bpo-39877: Deprecated PyEval_InitThreads() and PyEval_ThreadsInitialized(). Calling PyEval_InitThreads() now does nothing.

Reassigning back to libvirt-python
Comment 3 Daniel Berrangé 2020-11-12 11:25:40 UTC 
Those messages, while important to fix, are merely warnings so don't cause the build to fail.

The real problem shown in the logs is the test suite failure:


running test
/usr/bin/python3 sanitytest.py build/lib.linux-x86_64-3.1 /usr/share/libvirt/api/libvirt-api.xml
Traceback (most recent call last):
  File "/builddir/build/BUILD/libvirt-python-6.9.0/sanitytest.py", line 11, in <module>
    import libvirt
ModuleNotFoundError: No module named 'libvirt'
error: command '/usr/bin/python3' failed with exit 


It looks like we're truncating the version number to 3.1 when passing the build directory.
Comment 4 Miro Hrončok 2020-11-12 11:35:21 UTC 
From setup.py:

        plat_specifier = ".%s-%s" % (self.plat_name, sys.version[0:3])


I believe this is the cause. It evals to 3.1 on Python 3.10.
Comment 5 Daniel Berrangé 2020-11-12 15:09:49 UTC 
Fixes proposed upstream in https://gitlab.com/libvirt/libvirt-python/-/merge_requests/30
Comment 6 Miro Hrončok 2020-11-16 10:02:52 UTC 
I've patched this in our copr to unblock testing the depended packages. Let me know if I should submit a PR.

https://copr.fedorainfracloud.org/coprs/g/python/python3.10/build/1770405/
Comment 7 Daniel Berrangé 2020-11-16 10:05:32 UTC 
It'll get fixed in the next monthly rebase of libvirt around Dec 1st
Comment 8 Miro Hrončok 2020-11-16 10:32:05 UTC 
That'll work nicely, thanks Daniel.
Note You need to log in before you can comment on or make changes to this bug.