Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1813076 - pcp fails to build with Python 3.9 (Cython classmethod problem related)
Summary: pcp fails to build with Python 3.9 (Cython classmethod problem related)
Keywords:
Status: CLOSED DUPLICATE of bug 1843783
Alias: None
Product: Fedora
Classification: Fedora
Component: pcp
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Nathan Scott
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On: 1788506
Blocks: F33FTBFS PYTHON39
TreeView+ depends on / blocked
 
Reported: 2020-03-12 21:40 UTC by Miro Hrončok
Modified: 2020-06-07 18:23 UTC (History)
20 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-06-07 18:23:57 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Miro Hrončok 2020-03-12 21:40:27 UTC
pcp fails to build with Python 3.9.0a4:

make -f /usr/share/selinux/devel/Makefile
make[3]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.
Compiling targeted pcpupstream-docker module
Creating targeted pcpupstream-docker.pp policy package
Compiling targeted pcpupstream-container module
Creating targeted pcpupstream-container.pp policy package
Compiling targeted pcpupstream module
pcpupstream.te:383:ERROR 'permission map is not defined for class file' at token ';' on line 2593:
	allow pcp_domain file_type:file map;
#line 383
/usr/bin/checkmodule:  error(s) encountered while parsing configuration
make[3]: *** [/usr/share/selinux/devel/include/Makefile:157: tmp/pcpupstream.mod] Error 1
rm tmp/pcpupstream-docker.mod.fc tmp/pcpupstream-docker.mod tmp/pcpupstream-container.mod.fc tmp/pcpupstream-container.mod
make[2]: *** [GNUmakefile:37: pcpupstream.te] Error 2
make[1]: *** [GNUmakefile:133: default_pcp] Error 2
make[1]: Leaving directory '/builddir/build/BUILD/pcp-5.0.3/src'
make: *** [GNUmakefile:49: default_pcp] Error 2
error: Bad exit status from /var/tmp/rpm-tmp.Wpb2Yv (%build)
    Bad exit status from /var/tmp/rpm-tmp.Wpb2Yv (%build)

For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.9/fedora-rawhide-x86_64/01300028-pcp/

For all our attempts to build pcp with Python 3.9, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.9/package/pcp/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.9:
https://copr.fedorainfracloud.org/coprs/g/python/python3.9/

Let us know here if you have any questions.

Python 3.9 will be included in Fedora 33. To make that update smoother, we're building Fedora packages with early pre-releases of Python 3.9.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Nathan Scott 2020-03-12 23:47:16 UTC
Lukas, this failure looks like it may be fallout from recent changes in selinux-policy (outside of PCP)?  Any tips on resolution for us if so?  Thanks.

Comment 2 Petr Lautrbach 2020-03-13 20:51:28 UTC
The problem is at the begging of the build.log:

checking if selinux policy should be included... yes
Traceback (most recent call last):
  File "/usr/bin/seinfo", line 21, in <module>
    import setools
  File "/usr/lib64/python3.9/site-packages/setools/__init__.py", line 79, in <module>
    from .diff import PolicyDifference
  File "/usr/lib64/python3.9/site-packages/setools/diff/__init__.py", line 37, in <module>
    from .terules import TERulesDifference
  File "/usr/lib64/python3.9/site-packages/setools/diff/terules.py", line 201, in <module>
    class TERulesDifference(Difference):
  File "/usr/lib64/python3.9/site-packages/setools/diff/terules.py", line 208, in TERulesDifference
    diff_allows = av_diff_template("allow")
  File "/usr/lib64/python3.9/site-packages/setools/diff/terules.py", line 71, in av_diff_template
    ruletype = TERuletype.lookup(ruletype)
  File "setools/policyrep/util.pxi", line 59, in setools.policyrep.PolicyEnum.lookup
TypeError: __new__() missing 2 required positional arguments: 'bases' and 'classdict'


seinfo doesn't work an therefore pcp_selinux_hostname_exec_map is false and map permission for file class is not added to require {} block.

I've confirmed the setools issue on my system with @python/python3.9 enabled:

# rpm -qf /usr/bin/seinfo
setools-console-4.2.2-4.fc33.x86_64

# seinfo
Traceback (most recent call last):
  File "/usr/bin/seinfo", line 21, in <module>
    import setools
  File "/usr/lib64/python3.9/site-packages/setools/__init__.py", line 79, in <module>
    from .diff import PolicyDifference
  File "/usr/lib64/python3.9/site-packages/setools/diff/__init__.py", line 37, in <module>
    from .terules import TERulesDifference
  File "/usr/lib64/python3.9/site-packages/setools/diff/terules.py", line 201, in <module>
    class TERulesDifference(Difference):
  File "/usr/lib64/python3.9/site-packages/setools/diff/terules.py", line 208, in TERulesDifference
    diff_allows = av_diff_template("allow")
  File "/usr/lib64/python3.9/site-packages/setools/diff/terules.py", line 71, in av_diff_template
    ruletype = TERuletype.lookup(ruletype)
  File "setools/policyrep/util.pxi", line 59, in setools.policyrep.PolicyEnum.lookup
TypeError: __new__() missing 2 required positional arguments: 'bases' and 'classdict'

Comment 3 Fedora Admin XMLRPC Client 2020-03-23 16:43:49 UTC
This package has changed maintainer in the Fedora.
Reassigning to the new maintainer of this component.

Comment 4 Petr Lautrbach 2020-03-25 09:35:38 UTC
According to https://github.com/SELinuxProject/setools/issues/42#issuecomment-602854016 it seems to be a cython issue, as setools.policyrep.PolicyEnum.lookup is a @classmethod. There is no explicit __new__ method in the SETools code.

If it's not a cython issue, could you please provide me a guidance how to fix it?

Comment 5 Miro Hrončok 2020-03-25 10:58:05 UTC
Thanks for the investigation.

Let me keep this assigned to pcp so we can properly track the failure, but I have marked is as blocked by bz1788506. No action necessary here (for now).

Comment 6 Miro Hrončok 2020-05-21 09:19:19 UTC
Note that we have somewhat workarounded the Cython issue, but the recent builds in https://copr.fedorainfracloud.org/coprs/g/python/python3.9/package/pcp/ seem to fail for unrelated reasons.

Comment 7 Mark Goodwin 2020-05-22 00:13:15 UTC
Nathan, do recent pcp-5.1.1 commits in this area fix the build error (see log below)?

commit aeb081be7459df3528ec241ffe391336702e07ab
Author: Nathan Scott <nathans>
Date:   Tue May 12 10:42:38 2020 +1000

    selinux: add policy to allow openvswitch commands to run

commit 55937769c6ac5159d345dfd6d8e6aa7c33258ceb
Author: Nathan Scott <nathans>
Date:   Tue May 5 15:32:46 2020 +1000

    selinux: resolve bpftrace pmda access with latest fedora selinux policy



Most recent build log at https://copr-be.cloud.fedoraproject.org/results/%40python/python3.9/fedora-rawhide-x86_64/01396296-pcp/build.log.gz seems to still show the same or similar issue, i.e. 

During configure :
...
checking if selinux policy should be included... yes
<frozen importlib._bootstrap>:228: RuntimeWarning: builtins.type size changed, may indicate binary incompatibility. Expected 872 from C header, got 880 from PyObject
Traceback (most recent call last):
  File "/usr/bin/seinfo", line 21, in <module>
    import setools
  File "/usr/lib64/python3.9/site-packages/setools/__init__.py", line 81, in <module>
    from .diff import PolicyDifference
  File "/usr/lib64/python3.9/site-packages/setools/diff/__init__.py", line 39, in <module>
    from .terules import TERulesDifference
  File "/usr/lib64/python3.9/site-packages/setools/diff/terules.py", line 386, in <module>
    class TERulesDifference(Difference):
  File "/usr/lib64/python3.9/site-packages/setools/diff/terules.py", line 393, in TERulesDifference
    diff_allows = av_diff_template("allow")
  File "/usr/lib64/python3.9/site-packages/setools/diff/terules.py", line 229, in av_diff_template
    ruletype = TERuletype.lookup(ruletype)
  File "setools/policyrep/util.pxi", line 53, in setools.policyrep.PolicyEnum.lookup
TypeError: __new__() missing 2 required positional arguments: 'bases' and 'classdict'
checking for selinux files_mmap_all_files() macro... yes

And then later building the PCP selinux module :
...
make -f /usr/share/selinux/devel/Makefile
Compiling targeted pcpupstream-container module
Compiling targeted pcpupstream-docker module
Compiling targeted pcpupstream module
Creating targeted pcpupstream-container.pp policy package
Creating targeted pcpupstream-docker.pp policy package
pcpupstream.te:386:ERROR 'permission map is not defined for class file' at token ';' on line 2596:
	allow pcp_domain file_type:file map;
#line 386
/usr/bin/checkmodule:  error(s) encountered while parsing configuration
rm tmp/pcpupstream-docker.mod.fc tmp/pcpupstream.mod.fc tmp/pcpupstream-container.mod tmp/pcpupstream-container.mod.fc tmp/pcpupstream-docker.mod
make[3]: *** [/usr/share/selinux/devel/include/Makefile:157: tmp/pcpupstream.mod] Error 1
make[2]: *** [GNUmakefile:37: pcpupstream.te] Error 2
make[1]: *** [GNUmakefile:134: default_pcp] Error 2
make[1]: Leaving directory '/builddir/build/BUILD/pcp-5.1.0/src'
make: *** [GNUmakefile:49: default_pcp] Error 2
error: Bad exit status from /var/tmp/rpm-tmp.HquNUu (%build)
RPM build errors:
    Bad exit status from /var/tmp/rpm-tmp.HquNUu (%build)
Child return code was: 1

Comment 8 Miro Hrončok 2020-05-22 00:43:40 UTC
<frozen importlib._bootstrap>:228: RuntimeWarning: builtins.type size changed,
may indicate binary incompatibility. Expected 872 from C header, got 880 from
PyObject

this might mean that one or more of the dependencies was built with older Python 3.9 pre-release :/

I will check if that'S the case, tomorrow.

Comment 9 Nathan Scott 2020-05-22 00:44:57 UTC
(In reply to Mark Goodwin from comment #7)
> Nathan, do recent pcp-5.1.1 commits in this area fix the build error (see
> log below)?

I don't think so.

Comment 10 Miro Hrončok 2020-05-22 09:57:57 UTC
> RuntimeWarning: builtins.type size changed, may indicate binary incompatibility. Expected 872 from C header, got 880 from PyObject

The latest failure doesn't have this anymore.

Comment 11 Miro Hrončok 2020-05-25 13:01:36 UTC
This comment is mass posted to all bugs blocking the Python 3.9 tracker, sorry if it is not 100 % relevant. When in doubt, please ask.


The Python 3.9 rebuild is in progress in a Koji side tag.

If you fix this bug, please don't rebuild the package in regular rawhide, but do it in the side tag with:

    $ fedpkg build --target=f33-python

The rebuild is progressing slowly and it is possible this package won't have all the required build dependencies yet. If that's the case, please just leave the fix committed and pushed and we will eventually rebuild it for you.

You are not asked to go and try rebuild all the missing dependencies yourself. If you know there is a bootstrap loop in the dependencies, let me know and we can untangle it together.

If you want to test your fix or reproduce the failure, you can still use the Copr repo mentioned in the initial comment of this bug: https://copr.fedorainfracloud.org/coprs/g/python/python3.9/

Comment 12 Miro Hrončok 2020-05-29 07:11:35 UTC
Python 3.9 update: The f33-python side tag is currently being merged.

New builds in f33-python are no longer possible, but python3 is not yet updated to Python 3.9 in rawhide. You can check when Python is Python 3.9 with:

    $ koji wait-repo f33-build --build python3.9-3.9.0~b1-3.fc3

And build the packages normally after that.

Comment 13 Miro Hrončok 2020-06-07 18:23:57 UTC

*** This bug has been marked as a duplicate of bug 1843783 ***


Note You need to log in before you can comment on or make changes to this bug.