Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1841335
| Summary: | python-shapely fails to build on s390x, blocking the Python 3.9 rebuild of many packages | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Miro Hrončok <mhroncok> | ||||||
| Component: | geos | Assignee: | Sandro Mani <manisandro> | ||||||
| Status: | CLOSED RAWHIDE | QA Contact: | Fedora Extras Quality Assurance <extras-qa> | ||||||
| Severity: | high | Docs Contact: | |||||||
| Priority: | unspecified | ||||||||
| Version: | rawhide | CC: | dan, devrim, hannsj_uhl, igor.raits, jcp, jmlich83, loganjerry, manisandro, pviktori, tom, volker27 | ||||||
| Target Milestone: | --- | ||||||||
| Target Release: | --- | ||||||||
| Hardware: | Unspecified | ||||||||
| OS: | Unspecified | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | geos-3.8.1-2.fc33, python-shapely-1.7-3b1.fc33 | Doc Type: | If docs needed, set a value | ||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2020-06-17 09:38:02 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Bug Depends On: | |||||||||
| Bug Blocks: | 467765, 1750908, 1803234, 1785415, 1838465, 1841631, 1841684, 1841695, 1841702, 1841712, 1841788, 1841791, 1842049, 1842107, 1842156, 1842312 | ||||||||
| Attachments: |
|
||||||||
|
Description
Miro Hrončok
2020-05-28 21:48:07 UTC
Python 3.9 update: The f33-python side tag is currently being merged.
New builds in f33-python are no longer possible, but python3 is not yet updated to Python 3.9 in rawhide. You can check when Python is Python 3.9 with:
$ koji wait-repo f33-build --build python3.9-3.9.0~b1-3.fc3
And build the packages normally after that.
If I see right, then nothing changed in python-shapely, but the buildroot is different ... rebuild in F-31 = OK rebuild in F-32 -> same errors as with the rawhide/python-3.9 builds The python code didn't change, but geos did. The last successful build of python-shapely was on January 31. On March 2, geos 3.8.0 was built for F32+, followed later by geos 3.8.1. There may be an endianness bug in geos 3.8.x. Or something changed bit width, and the shapely cython code needs to be updated. Please always regenerate Cython code on build. I sent a PR for it: https://bugzilla.redhat.com/show_bug.cgi?id=1841335 Sadly, it doesn't help the FTBFS :( The PR link is: https://src.fedoraproject.org/rpms/python-shapely/pull-request/5 *** Bug 1842150 has been marked as a duplicate of this bug. *** Created attachment 1693859 [details]
Demonstration of geos bug
The attached file is a C translation of one of the failing python-shapely tests. It shows that the bug is in geos, not python-shapely. Build it with "gcc -o test test.c -lgeos_c". If built and run on a little endian machine, it correctly produces this output:
TopologyException: side location conflict at 459 346
The result is 2
If built and run on s390x, it incorrectly produces this output:
The result is 0
Created attachment 1693904 [details]
C++ version of demonstration
This version demonstrates that the bug is in the C++ core of geos, not in the C wrapper that python-shapely uses. Build it like this: "g++ -o test test.cpp -lgeos". You will see warnings, but they can be ignored.
Hello, Please note that this comment was generated automatically. If you feel that this output has mistakes, please contact me via email (ignatenkobrain). All subpackages of a package agaisnt which this bug was filled are now installable or removed from Fedora 33. Thanks for taking care of it! Can we temporarily skip the python-shapely 3 failing shapely tests on s390x to unblock handful of packages that might not be affected by this problem? Since geos is computing incorrect results on s390x, then python-shapely is too. That means that anything sitting on top of python-shapely is likely to fail its test suite on s390x, too, I would think. We should concentrate our efforts on diagnosing and fixing the geos problem. In fact, I tried to do just that, but ran into bug 1842325. If somebody can fix that, I think we should be able to fix the geos bug pretty quickly. Note that I am not a maintainer for either geos or python-shapely. I'm just Random Drive-by User who is trying to lend a hand. I've noticed two things: 1) The geos spec file does this: %ifarch armv7hl aarch64 s390x ppc64le make test || : %else make test %endif and indeed, many tests fail on s390x. 2) The reason I can't get the C++ pretty printers to work for me so I can debug this issue is that python support was compiled out of gdb due to bug 1829702. 2) Can you debug this on Fedora 32? Latest geos git still suffers from test failures [1]. Any chance of getting ssh access to a s390x machine to bisect this? [1] https://koji.fedoraproject.org/koji/taskinfo?taskID=45318460 (In reply to Sandro Mani from comment #15) > Latest geos git still suffers from test failures [1]. > > Any chance of getting ssh access to a s390x machine to bisect this? > > [1] https://koji.fedoraproject.org/koji/taskinfo?taskID=45318460 I can arrange access to s390x, send me an email to sharkcz @Dan Horák Have you received my PM? (In reply to Sandro Mani from comment #17) > @Dan Horák Have you received my PM? yes, and I have replied the other day, please check your spam box ... Hmm don't see it anywhere. Can you please resend? (In reply to Sandro Mani from comment #19) > Hmm don't see it anywhere. Can you please resend? done, sender is dan at danny dot cz FTR: I'm trying to bisect this trough Koji. This is what I got:
10ff0f8d6809c212aa9eb75f710bd068a90ccff0 is the first new commit
commit 10ff0f8d6809c212aa9eb75f710bd068a90ccff0
Author: Paul Ramsey <pramsey>
Date: Fri Dec 14 09:24:28 2018 -0800
Remove references to RobustDeterminant
JTS 2fc9fa215cdfff4210ec55cb8c6cfb79618b4ff2
src/algorithm/CGAlgorithmsDD.cpp | 5 +++++
src/algorithm/RayCrossingCounter.cpp | 3 ++-
src/algorithm/SIRtreePointInRing.cpp | 3 ++-
3 files changed, 9 insertions(+), 2 deletions(-)
I am not available to inspect the commit at this point.
(the commit does not rever cleanly on 3.8.1) I'm also looking into it, the problem is that newly added tests were immediately failing on those arches, so the actual code may have already been broken on those arches way before... Note that the bisected commit is the one where the test.c reproducer from Jerry starts to give incorrect results on s390x. I.e. I was not bisecting based on unit test results, but only on this one case. Likely the DD class [1][2] itself being the issue... [1] https://github.com/libgeos/geos/blob/master/include/geos/math/DD.h [2] https://github.com/libgeos/geos/blob/master/src/math/DD.cpp I've summarized this to https://github.com/libgeos/geos/issues/323 but that is only a mirror. Upstream uses https://trac.osgeo.org/geos/report/1 where I requested an account (pending approval). (In reply to Sandro Mani from comment #15) > Latest geos git still suffers from test failures [1]. > > Any chance of getting ssh access to a s390x machine to bisect this? > > [1] https://koji.fedoraproject.org/koji/taskinfo?taskID=45318460 bed36f15c780057ae9b83eb9cd2e8ef6a9ada498 is the fixer of the C reproducer. Not sure if it fixes shapely. Latest goes git still fails tests, but might actually fix shapely tests. I've opened https://src.fedoraproject.org/rpms/geos/pull-request/2 Could somebody with access to s390x verify it fixes shapely tests? Alternatively, Dan, could you e-mail me the access details? It looks good, I have used F-32, rebuilt geos with the PR #2, installed the rpms, then built shapely, all tests passed. Thanks Miro for taking care of this and sorry for not being more responsive, dayjob is keeping me pretty busy at the moment. Thank you all. |