Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1890878 - libxml2 fails to build with Python 3.10: SystemError: PY_SSIZE_T_CLEAN macro must be defined for '#' formats
Summary: libxml2 fails to build with Python 3.10: SystemError: PY_SSIZE_T_CLEAN macro ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: libxml2
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Igor Raits
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: PYTHON3.10
TreeView+ depends on / blocked
 
Reported: 2020-10-23 07:49 UTC by Tomáš Hrnčiar
Modified: 2021-09-29 13:17 UTC (History)
7 users (show)

Fixed In Version: libxml2-2.9.10-9.fc34 libxml2-2.9.10-9.eln105
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-09-29 12:54:36 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Tomáš Hrnčiar 2020-10-23 07:49:36 UTC
libxml2 fails to build with Python 3.10.0a1.

make[2]: Leaving directory '/builddir/build/BUILD/libxml2-2.9.10/py3/python/tests'
test.xml:1: parser error : Couldn't find end of Start Tag foo
<foo
    ^
SystemError: PY_SSIZE_T_CLEAN macro must be defined for '#' formats

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/builddir/build/BUILD/libxml2-2.9.10/py3/python/tests/../../../python/tests/push.py", line 9, in <module>
    ctxt.parseChunk("/>", 2, 1)
  File "/builddir/build/BUILD/libxml2-2.9.10/python/libxml2.py", line 5101, in parseChunk
    ret = libxml2mod.xmlParseChunk(self._o, chunk, size, terminate)
SystemError: <built-in function xmlParseChunk> returned a result with an error set

For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.10/fedora-rawhide-x86_64/01715877-libxml2/

For all our attempts to build libxml2 with Python 3.10, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.10/package/libxml2/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.10:
https://copr.fedorainfracloud.org/coprs/g/python/python3.10/

Let us know here if you have any questions.

Python 3.10 will be included in Fedora 35. To make that update smoother, we're building Fedora packages with early pre-releases of Python 3.10.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Miro Hrončok 2020-10-23 08:18:10 UTC
https://docs.python.org/3.10/whatsnew/3.10.html#id2

The PY_SSIZE_T_CLEAN macro must now be defined to use PyArg_ParseTuple() and Py_BuildValue() formats which use #: es#, et#, s#, u#, y#, z#, U# and Z#. See Parsing arguments and building values and the PEP 353.

https://docs.python.org/3.10/c-api/arg.html#arg-parsing
https://www.python.org/dev/peps/pep-0353
https://bugs.python.org/issue40943

Comment 2 Victor Stinner 2020-11-09 17:26:11 UTC
I reported the issue to upstream: https://gitlab.gnome.org/GNOME/libxml2/-/issues/203

And I proposed a fix: https://gitlab.gnome.org/GNOME/libxml2/-/merge_requests/87

Comment 3 Victor Stinner 2020-11-10 14:02:01 UTC
I proposed to https://src.fedoraproject.org/rpms/libxml2/pull-request/10 to backport my fix and fix the package (always regenerate the 3 Python generated files).

Comment 5 Fedora Update System 2020-11-12 11:54:15 UTC
FEDORA-2020-576ed43594 has been pushed to the Fedora 34 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 6 Fedora Update System 2020-11-12 12:09:16 UTC
FEDORA-2020-160ee73cea has been pushed to the Fedora ELN stable repository.
If problem still persists, please make note of it in this bug report.

Comment 7 Victor Stinner 2020-11-19 20:17:22 UTC
> I proposed to https://src.fedoraproject.org/rpms/libxml2/pull-request/10 to backport my fix and fix the package (always regenerate the 3 Python generated files).

FYI my PR has been merged upstream:
* https://gitlab.gnome.org/GNOME/libxml2/-/commit/f42a0524c693a6d52e95a37c9cf04df1be48262c
* https://gitlab.gnome.org/GNOME/libxml2/-/commit/ac5e99911a45ad6f57a6aa7b33fd2de9da72aa72

Comment 8 Lukas Javorsky 2021-09-29 10:23:45 UTC
Hi, I've encountered this issue on Fedora 35 container.

This issue occurs when I try to build the module build for RHEL-8 using this command:
rhpkg module-build --scratch --buildrequires platform:el8.5.0

The Fedora container that I'm using is here:
https://gitlab.cee.redhat.com/platform-eng-core-services/core-services-dockerfiles/-/tree/master/pkg-container

Error message: Could not execute module_build: PY_SSIZE_T_CLEAN macro must be defined for '#' formats

PS: I've tried it in the Fedora 34 container and it worked as expected, so the issue must be in the commits between these two versions

Reopening this BZ, if you prefer to create a new one let me know.

Comment 9 Miro Hrončok 2021-09-29 10:36:25 UTC
I am not sure that gitlab.cee link works outside of the Red Hat network and this is  Fedora Bugzilla. Could you provide a more accessible reproducer, please?


Also, what makes you think that the error comes from libxml2?

Comment 10 Miro Hrončok 2021-09-29 10:39:04 UTC
A list of installed packages (including their versions, releaes and architecture) would be helpful to determine what packages could cause this. Could you please attach the output of `rpm -qa`?

Also, is there a way to see the full traceback?

Comment 12 Miro Hrončok 2021-09-29 12:46:10 UTC
> Also, is there a way to see the full traceback?

Please, run again with rhpkg --debug. That raises the exception instead of hiding it:

    except Exception as e:
        ...

        if getattr(client.args, 'debug', False):
            raise
        log.error('Could not execute %s: %s' % (client.args.command.__name__, e))
        sys.exit(1)

Comment 13 Miro Hrončok 2021-09-29 12:54:36 UTC
I've got it:

Traceback (most recent call last):
  File "/usr/bin/rhpkg", line 16, in <module>
    main()
  File "/usr/lib/python3.10/site-packages/rhpkg/__main__.py", line 104, in main
    sys.exit(client.args.command())
  File "/usr/lib/python3.10/site-packages/pyrpkg/cli.py", line 2296, in module_build
    build_ids = self._cmd.module_submit_build(
  File "/usr/lib/python3.10/site-packages/pyrpkg/__init__.py", line 3863, in module_submit_build
    resp = self.module_send_authorized_request(
  File "/usr/lib/python3.10/site-packages/pyrpkg/__init__.py", line 3786, in module_send_authorized_request
    resp = requests.request(verb, url, data=data, auth=auth, **kwargs)
  File "/usr/lib/python3.10/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python3.10/site-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3.10/site-packages/requests/sessions.py", line 662, in send
    r = dispatch_hook('response', hooks, r, **kwargs)
  File "/usr/lib/python3.10/site-packages/requests/hooks.py", line 31, in dispatch_hook
    _hook_data = hook(hook_data, **kwargs)
  File "/usr/lib/python3.10/site-packages/requests_kerberos/kerberos_.py", line 385, in handle_response
    self.cbt_struct = kerberos.channelBindings(application_data=cbt_application_data)
SystemError: PY_SSIZE_T_CLEAN macro must be defined for '#' formats



A reproducer:

>>> import kerberos
>>> kerberos.__file__
'/usr/lib64/python3.10/site-packages/kerberos.cpython-310-x86_64-linux-gnu.so'
>>> kerberos.channelBindings(application_data={})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
SystemError: PY_SSIZE_T_CLEAN macro must be defined for '#' formats

Comment 15 Miro Hrončok 2021-09-29 13:17:10 UTC
Reported as bz2008899.


Note You need to log in before you can comment on or make changes to this bug.