1817665 – libmodulemd fails to build with Python 3.9: Memory leaks detected

Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.

Bug 1817665 - libmodulemd fails to build with Python 3.9: Memory leaks detected

Summary: libmodulemd fails to build with Python 3.9: Memory leaks detected

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Fedora
Classification:	Fedora
Component:	libmodulemd
Sub Component:
Version:	rawhide
Hardware:	Unspecified
OS:	Unspecified
Priority:	unspecified
Severity:	unspecified
Target Milestone:	---
Assignee:	Merlin Mathesius
QA Contact:	Fedora Extras Quality Assurance
Docs Contact:
URL:
Whiteboard:
Depends On:	1817649
Blocks:	PYTHON39
TreeView+	depends on / blocked

Reported:	2020-03-26 18:12 UTC by Miro Hrončok
Modified:	2020-04-25 02:19 UTC (History)
CC List:	8 users (show)
Fixed In Version:	2.9.2-3.fc33 libmodulemd-2.9.3-1.fc31 libmodulemd-2.9.3-1.fc30 libmodulemd-2.9.3-1.fc32
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:	2020-04-16 22:40:49 UTC
Type:	Bug
Embargoed:
Dependent Products:

Attachments	(Terms of Use)

Description Miro Hrončok 2020-03-26 18:12:22 UTC

libmodulemd fails to build with Python 3.9.0a5.

Traceback (most recent call last):
  File "/usr/bin/g-ir-scanner", line 103, in <module>
    from giscanner.scannermain import scanner_main
  File "/usr/lib64/gobject-introspection/giscanner/scannermain.py", line 36, in <module>
    from giscanner.dumper import compile_introspection_binary
  File "/usr/lib64/gobject-introspection/giscanner/dumper.py", line 28, in <module>
    from .gdumpparser import IntrospectionBinary
  File "/usr/lib64/gobject-introspection/giscanner/gdumpparser.py", line 25, in <module>
    from xml.etree.cElementTree import parse
ModuleNotFoundError: No module named 'xml.etree.cElementTree'

This is caused directly by bz1817649.


For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.9/fedora-rawhide-x86_64/01321057-libmodulemd/

For all our attempts to build libmodulemd with Python 3.9, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.9/package/libmodulemd/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.9:
https://copr.fedorainfracloud.org/coprs/g/python/python3.9/

Let us know here if you have any questions.

Python 3.9 will be included in Fedora 33. To make that update smoother, we're building Fedora packages with early pre-releases of Python 3.9.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Fedora Blocker Bugs Application 2020-03-26 19:57:19 UTC

Proposed as a Blocker for 32-final by Fedora user sgallagh using the blocker tracking app because:

 "All elements of the default panel (or equivalent) configuration in all release-blocking desktops must function correctly in typical use." -- Final Criteria

I argue that screen-lock is (or at least should be!) typical use.

Comment 2 Miro Hrončok 2020-03-26 20:05:59 UTC

I believe the previous comment was supposed to be posted to a different bugzilla.

Comment 3 Stephen Gallagher 2020-03-26 21:17:01 UTC

Oops, too many tabs.

Comment 4 Miro Hrončok 2020-03-26 23:25:57 UTC

Even with bz1817649 fixed, libmodulemd still fails to build.

The output from the failed tests:

64/64 valgrind                                FAIL    465.86681604385376 s (exit status (1,))

--- command ---
22:40:40 MESON_SOURCE_ROOT='/builddir/build/BUILD/modulemd-2.9.2' G_MESSAGES_DEBUG='all' MESON_BUILD_ROOT='/builddir/build/BUILD/modulemd-2.9.2/x86_64-redhat-linux-gnu' LC_ALL='C' G_DEBUG='fatal-warnings,fatal-criticals' TEST_DATA_PATH='/builddir/build/BUILD/modulemd-2.9.2/modulemd/tests/test_data' /usr/bin/python3 /builddir/build/BUILD/modulemd-2.9.2/x86_64-redhat-linux-gnu/../modulemd/tests/test-valgrind.py buildopts_debug component_module_debug component_rpm_debug compression_debug defaults_debug defaultsv1_debug dependencies_debug module_debug module_index_debug module_index_merger_debug modulestream_debug profile_debug rpm_map_debug service_level_debug translation_debug translation_entry_debug buildopts_python3_debug componentrpm_python3_debug defaults_python3_debug defaultsv1_python3_debug dependencies_python3_debug merger_python3_debug module_python3_debug moduleindex_python3_debug modulepackager_python3_debug modulestream_python3_debug profile_python3_debug rpmmap_python3_debug servicelevel_python3_debug translation_python3_debug translationentry_python3_debug
--- Listing only the last 100 lines from a long log. ---
    <frame>
      <ip>0x4B1CCCC</ip>
      <obj>/usr/lib64/libpython3.9.so.1.0</obj>
      <fn>_PyObject_MakeTpCall</fn>
    </frame>
    <frame>
      <ip>0x4B8A02D</ip>
      <obj>/usr/lib64/libpython3.9.so.1.0</obj>
    </frame>
    <frame>
      <ip>0x4B1C350</ip>
      <obj>/usr/lib64/libpython3.9.so.1.0</obj>
    </frame>
    <frame>
      <ip>0x4B1C179</ip>
      <obj>/usr/lib64/libpython3.9.so.1.0</obj>
    </frame>
    <frame>
      <ip>0x4B14998</ip>
      <obj>/usr/lib64/libpython3.9.so.1.0</obj>
      <fn>_PyEval_EvalFrameDefault</fn>
    </frame>
    <frame>
      <ip>0x4B13590</ip>
      <obj>/usr/lib64/libpython3.9.so.1.0</obj>
    </frame>
  </stack>
</error>

<errorcounts>
</errorcounts>

<suppcounts>
  <pair>
    <count>734</count>
    <name>Handle PyMalloc confusing valgrind</name>
  </pair>
  <pair>
    <count>615</count>
    <name>Handle PyMalloc confusing valgrind</name>
  </pair>
  <pair>
    <count>33</count>
    <name>g-type-register-static</name>
  </pair>
  <pair>
    <count>6</count>
    <name>Handle PyMalloc confusing valgrind</name>
  </pair>
  <pair>
    <count>8</count>
    <name>g-type-register-static-realloc</name>
  </pair>
  <pair>
    <count>129</count>
    <name>g-type-register-static-calloc</name>
  </pair>
  <pair>
    <count>57</count>
    <name>g-type-register-fundamental-calloc</name>
  </pair>
  <pair>
    <count>1</count>
    <name>g_strerror</name>
  </pair>
  <pair>
    <count>1</count>
    <name>Handle PyMalloc confusing valgrind</name>
  </pair>
  <pair>
    <count>17</count>
    <name>g-type-register-fundamental</name>
  </pair>
  <pair>
    <count>1</count>
    <name>Python3 Unicode uninitialized value</name>
  </pair>
</suppcounts>

</valgrindoutput>


--- stderr ---
Memory leak detected in buildopts_python3_debug
Memory leak detected in componentrpm_python3_debug
Memory leak detected in defaults_python3_debug
Memory leak detected in defaultsv1_python3_debug
Memory leak detected in dependencies_python3_debug
Memory leak detected in merger_python3_debug
Memory leak detected in module_python3_debug
Memory leak detected in moduleindex_python3_debug
Memory leak detected in modulepackager_python3_debug
Memory leak detected in modulestream_python3_debug
Memory leak detected in profile_python3_debug
Memory leak detected in rpmmap_python3_debug
Memory leak detected in servicelevel_python3_debug
Memory leak detected in translation_python3_debug
Memory leak detected in translationentry_python3_debug
-------


For the build logs, see:
https://download.copr.fedorainfracloud.org/results/@python/python3.9/fedora-rawhide-x86_64/01321354-libmodulemd/

For all our attempts to build libmodulemd with Python 3.9, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.9/package/libmodulemd/


CCing Victor and Petr. Is this a leak in Python itself?

Comment 5 Stephen Gallagher 2020-03-27 12:46:48 UTC

Merlin, could you investigate and see if this is our bug or Python's?

Please attach the standard valgrind output to this ticket (as opposed to the XML version above).

Comment 6 Merlin Mathesius 2020-03-27 17:30:02 UTC

I had valgrind generate suppression entries for each of the definite memory leaks it found with Python 3.9. All 30 of them are exactly the same:

{
   <insert_a_suppression_name_here>
   Memcheck:Leak
   match-leak-kinds: definite
   fun:malloc
   fun:PyThread_allocate_lock
   obj:/usr/lib64/libpython3.9.so.1.0
   obj:/usr/lib64/libpython3.9.so.1.0
   fun:Py_InitializeFromConfig
   obj:/usr/lib64/libpython3.9.so.1.0
   obj:/usr/lib64/libpython3.9.so.1.0
   fun:Py_BytesMain
   fun:(below main)
}

Comment 7 Miro Hrončok 2020-03-27 18:19:27 UTC

See also https://github.com/python/cpython/blob/master/Misc/README.valgrind
And https://github.com/python/cpython/blob/master/Misc/valgrind-python.supp

{
   Suppress leaking the GIL.  Happens once per process, see comment in ceval.c.
   Memcheck:Leak
   fun:malloc
   fun:PyThread_allocate_lock
   fun:PyEval_InitThreads
}

{
   Suppress leaking the GIL after a fork.
   Memcheck:Leak
   fun:malloc
   fun:PyThread_allocate_lock
   fun:PyEval_ReInitThreads
}

Comment 8 Merlin Mathesius 2020-03-27 20:47:44 UTC

https://github.com/fedora-modularity/libmodulemd/pull/464 has been created to add the valgrind suppression for just the leak that has been observed.

Comment 9 Victor Stinner 2020-03-28 00:57:42 UTC

> https://github.com/fedora-modularity/libmodulemd/pull/464 has been created to add the valgrind suppression for just the leak that has been observed.

I reviewed this change, it LGTM.

Comment 10 Miro Hrončok 2020-04-01 14:15:10 UTC

Thanks for the fix.

Comment 11 Stephen Gallagher 2020-04-01 14:46:09 UTC

Not yet in Rawhide; waiting on gating tests.

Comment 12 Fedora Update System 2020-04-01 14:46:33 UTC

FEDORA-2020-41918baa5a has been submitted as an update to Fedora 33. https://bodhi.fedoraproject.org/updates/FEDORA-2020-41918baa5a

Comment 13 Miro Hrončok 2020-04-01 15:05:47 UTC

Technically, Python 3.9 is not in rawhide either, https://copr.fedorainfracloud.org/coprs/g/python/python3.9/package/libmodulemd/ is however fixed.

Comment 14 Fedora Update System 2020-04-01 15:46:55 UTC

FEDORA-2020-59b62d94c7 has been submitted as an update to Fedora 33. https://bodhi.fedoraproject.org/updates/FEDORA-2020-59b62d94c7

Comment 15 Fedora Update System 2020-04-08 16:09:58 UTC

FEDORA-2020-34b22b690d has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2020-34b22b690d

Comment 16 Fedora Update System 2020-04-08 16:09:59 UTC

FEDORA-2020-5f5da0cd13 has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2020-5f5da0cd13

Comment 17 Fedora Update System 2020-04-08 16:10:00 UTC

FEDORA-2020-9268872640 has been submitted as an update to Fedora 32. https://bodhi.fedoraproject.org/updates/FEDORA-2020-9268872640

Comment 18 Fedora Update System 2020-04-09 18:38:10 UTC

FEDORA-2020-9268872640 has been pushed to the Fedora 32 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-9268872640`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-9268872640

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 19 Fedora Update System 2020-04-09 19:13:43 UTC

FEDORA-2020-34b22b690d has been pushed to the Fedora 31 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-34b22b690d`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-34b22b690d

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 20 Fedora Update System 2020-04-09 20:14:18 UTC

FEDORA-2020-5f5da0cd13 has been pushed to the Fedora 30 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-5f5da0cd13`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-5f5da0cd13

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 21 Fedora Update System 2020-04-16 22:40:49 UTC

FEDORA-2020-34b22b690d has been pushed to the Fedora 31 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 22 Fedora Update System 2020-04-18 02:31:34 UTC

FEDORA-2020-5f5da0cd13 has been pushed to the Fedora 30 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 23 Fedora Update System 2020-04-25 02:19:26 UTC

FEDORA-2020-9268872640 has been pushed to the Fedora 32 stable repository.
If problem still persists, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.