Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1791969 - py4j version 0.10.9 / fails to build with Python 3.9: imports abc from collections
Summary: py4j version 0.10.9 / fails to build with Python 3.9: imports abc from collec...
Keywords:
Status: CLOSED DEFERRED
Alias: None
Product: Fedora
Classification: Fedora
Component: py4j
Version: rawhide
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
Assignee: Raphael Groner
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: PYTHON39
TreeView+ depends on / blocked
 
Reported: 2020-01-16 19:16 UTC by Miro Hrončok
Modified: 2020-02-28 09:07 UTC (History)
5 users (show)

Fixed In Version: py4j-0.10.9-1.fc32
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-02-28 09:07:40 UTC
Type: Bug
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github bartdag py4j issues 345 0 None closed py4j/java_collections.py:13: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collect... 2020-09-11 16:25:56 UTC

Description Miro Hrončok 2020-01-16 19:16:09 UTC 
py4j fails to build with Python 3.9.0a2.

Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/sphinx/config.py", line 361, in eval_config_file
    execfile_(filename, namespace)
  File "/usr/lib/python3.9/site-packages/sphinx/util/pycompat.py", line 81, in execfile_
    exec(code, _globals)
  File "/builddir/build/BUILD/py4j-0.10.8.1/py4j-web/conf.py", line 22, in <module>
    import py4j.java_gateway
  File "/builddir/build/BUILD/py4j-0.10.8.1/py4j-python/src/py4j/java_gateway.py", line 2496, in <module>
    __import__("py4j.java_collections")
  File "/builddir/build/BUILD/py4j-0.10.8.1/py4j-python/src/py4j/java_collections.py", line 13, in <module>
    from collections import (
ImportError: cannot import name 'MutableMapping' from 'collections' (/usr/lib64/python3.9/collections/__init__.py)

See https://docs.python.org/3.9/whatsnew/3.9.html#removed

"The abstract base classes in collections.abc no longer are exposed in the regular collections module. This will help create a clearer distinction between the concrete classes and the abstract base classes."


For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.9/fedora-rawhide-x86_64/01148667-py4j/

For all our attempts to build py4j with Python 3.9, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.9/package/py4j/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.9:
https://copr.fedorainfracloud.org/coprs/g/python/python3.9/

Let us know here if you have any questions.

Python 3.9 will be included in Fedora 33. To make that update smoother, we're building Fedora packages with early pre-releases of Python 3.9.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.
Comment 1 Hugo van Kemenade 2020-01-20 18:27:52 UTC 
This has been fixed in master in July 2019 (PR https://github.com/bartdag/py4j/pull/356) but the last release 0.10.8.1 was in October 2018 (https://github.com/bartdag/py4j/releases/tag/0.10.8.1).

I've asked for a new release in the issue: https://github.com/bartdag/py4j/issues/345.
Comment 2 Hugo van Kemenade 2020-01-25 17:44:13 UTC 
This has been released in version 0.10.9: https://github.com/bartdag/py4j/issues/345#issuecomment-578414971.
Comment 3 Raphael Groner 2020-02-06 23:47:58 UTC 
(In reply to Hugo van Kemenade from comment #2)
> This has been released in version 0.10.9:
> https://github.com/bartdag/py4j/issues/345#issuecomment-578414971.

Thanks for the hint. It's not yet released.
https://github.com/bartdag/py4j/commit/3745e493ee8ece0279a57683bbe3784db785b94e
Comment 4 Hugo van Kemenade 2020-02-08 15:41:31 UTC 
> Thanks for the hint. It's not yet released.
> https://github.com/bartdag/py4j/commit/
> 3745e493ee8ece0279a57683bbe3784db785b94e

0.10.9 was released Jan 25, 2020:

https://pypi.org/project/py4j/0.10.9/

They forgot to push the tags, but it's now done:

https://github.com/bartdag/py4j/issues/345#issuecomment-583247140
https://github.com/bartdag/py4j/releases/tag/0.10.9
Comment 5 Raphael Groner 2020-02-08 18:07:26 UTC 
Tests execution in mock gets stuck after:

     [exec] testPythonToJava (py4j.tests.memory_leak_test.ClientServerTest) ... ok
     [exec] testPythonToJavaToPython (py4j.tests.memory_leak_test.ClientServerTest) ... ok
Comment 6 Raphael Groner 2020-02-08 18:58:52 UTC 
Successfully built in rawhide.
Comment 7 Ben Cotton 2020-02-11 16:36:10 UTC 
This bug appears to have been reported against 'rawhide' during the Fedora 32 development cycle.
Changing version to 32.
Comment 8 Raphael Groner 2020-02-15 10:02:04 UTC 
Please try to rebuild with recent rawhide and feel free to reopen this bug if there's still an issue or you'd like to see builds for other branches.
Comment 9 Marcel Plch 2020-02-17 13:19:05 UTC 
There still is an issue, reopening.

Seems like some network test is failing:

    [junit] Received temp: null
    [junit] Testsuite: py4j.GatewayServerTest
    [junit] Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.813 sec
    [junit] 
    [junit] Testcase: testListener(py4j.GatewayServerTest):	Caused an ERROR
    [junit] null
    [junit] py4j.Py4JNetworkException
    [junit] 	at py4j.GatewayServer.startSocket(GatewayServer.java:789)
    [junit] 	at py4j.GatewayServer.start(GatewayServer.java:764)
    [junit] 	at py4j.GatewayServer.start(GatewayServer.java:747)
    [junit] 	at py4j.GatewayServerTest.testListener(GatewayServerTest.java:82)
    [junit] Caused by: java.net.BindException: Address already in use (Bind failed)
    [junit] 	at java.net.PlainSocketImpl.socketBind(Native Method)
    [junit] 	at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387)
    [junit] 	at java.net.ServerSocket.bind(ServerSocket.java:390)
    [junit] 	at py4j.GatewayServer.startSocket(GatewayServer.java:787)
    [junit] 
    [junit] 
BUILD FAILED
/builddir/build/BUILD/py4j-0.10.9/py4j-java/build.xml:141: Test py4j.GatewayServerTest failed

For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.9/fedora-rawhide-x86_64/01242346-py4j/
Comment 10 Raphael Groner 2020-02-18 16:52:28 UTC 
> [junit] Caused by: java.net.BindException: Address already in use (Bind failed)

Well known issue when execution is done of several tests in parallel, in optimization concerns. Will take a closer look.
Comment 11 Miro Hrončok 2020-02-28 09:07:40 UTC 
In Python 3.9.0a4 we have managed to revert the change that disables importing abstract base classes (collections.abc) directly form the regular collections module. It was reverted to ease transition for projects that did not yet actually dropped support for Python 2.7. The removal is scheduled for Python 3.10 instead.

We are mass closing all the bug reports about build failures that were caused by this. Expect a new Bugzilla report in case this package fails to build with Python 3.9 for a different reason as well.

In case you haven't done this already, we highly recommend to talk to your upstream and get this fixed anyway, because this will return in Python 3.10 and Python upstream is very unlikely to postpone this once more. In case you've already done that, thank you (this message is mass posted to all the relevant Bugzillas).
Note You need to log in before you can comment on or make changes to this bug.