Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1863830 - grep: FTBFS in Fedora rawhide/f33
Summary: grep: FTBFS in Fedora rawhide/f33
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: grep
Version: 33
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Jaroslav Škarvada
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: F33FTBFS 1872913 1872922
TreeView+ depends on / blocked
 
Reported: 2020-08-03 17:29 UTC by Fedora Release Engineering
Modified: 2020-09-15 20:48 UTC (History)
7 users (show)

Fixed In Version: grep-3.4-5.fc34 grep-3.4-5.fc33 grep-3.4-5.eln103
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-09-02 19:48:40 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
build.log (deleted)
2020-08-03 17:29 UTC, Fedora Release Engineering
no flags Details
root.log (deleted)
2020-08-03 17:29 UTC, Fedora Release Engineering
no flags Details
state.log (deleted)
2020-08-03 17:29 UTC, Fedora Release Engineering
no flags Details

Description Fedora Release Engineering 2020-08-03 17:29:51 UTC
grep failed to build from source in Fedora rawhide/f33

https://koji.fedoraproject.org/koji/taskinfo?taskID=47978688


For details on the mass rebuild see:

https://fedoraproject.org/wiki/Fedora_33_Mass_Rebuild
Please fix grep at your earliest convenience and set the bug's status to
ASSIGNED when you start fixing it. If the bug remains in NEW state for 8 weeks,
grep will be orphaned. Before branching of Fedora 34,
grep will be retired, if it still fails to build.

For more details on the FTBFS policy, please visit:
https://fedoraproject.org/wiki/Fails_to_build_from_source

Comment 1 Fedora Release Engineering 2020-08-03 17:29:53 UTC
Created attachment 1704969 [details]
build.log

file build.log too big, will only attach last 32768 bytes

Comment 2 Fedora Release Engineering 2020-08-03 17:29:54 UTC
Created attachment 1704970 [details]
root.log

file root.log too big, will only attach last 32768 bytes

Comment 3 Fedora Release Engineering 2020-08-03 17:29:55 UTC
Created attachment 1704971 [details]
state.log

Comment 4 Ben Cotton 2020-08-11 14:08:51 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 33 development cycle.
Changing version to 33.

Comment 5 Adam Williamson 2020-08-26 23:22:49 UTC
It seems that some of the tests are crashing on armv7hl and ppc64le. On armv7hl, the tests test-perror2 and test-strerror_r reliably crash. On ppc64le, the test test-float reliably crashes. This was not introduced by grep 3.4 - the same happens if you try to build grep 3.3 on current F33, and the crashes *don't* happen if you build grep 3.4 on F32. So the crashes have been triggered by something in the build chain being updated in F33+, most likely GCC or glibc or something.

Comment 6 Florian Weimer 2020-08-27 07:46:12 UTC
The test-perror2 failure reproduces under valgrind on x86-64:

==20== Invalid read of size 1
==20==    at 0x483DAB4: strcmp (vg_replace_strmem.c:847)
==20==    by 0x109414: main (test-perror2.c:84)
==20==  Address 0x4a1a3d0 is 0 bytes inside a block of size 17 free'd
==20==    at 0x483A9F5: free (vg_replace_malloc.c:538)
==20==    by 0x48E2134: strerror_l (in /usr/lib64/libc-2.32.so)
==20==    by 0x109328: main (test-perror2.c:72)
==20==  Block was alloc'd at
==20==    at 0x4839809: malloc (vg_replace_malloc.c:307)
==20==    by 0x48CA03F: __vasprintf_internal (in /usr/lib64/libc-2.32.so)
==20==    by 0x48A46F9: asprintf (in /usr/lib64/libc-2.32.so)
==20==    by 0x48E2184: strerror_l (in /usr/lib64/libc-2.32.so)
==20==    by 0x1092E2: main (test-perror2.c:67)
==20== 
==20== Invalid read of size 1
==20==    at 0x483DAC8: strcmp (vg_replace_strmem.c:847)
==20==    by 0x109414: main (test-perror2.c:84)
==20==  Address 0x4a1a3d1 is 1 bytes inside a block of size 17 free'd
==20==    at 0x483A9F5: free (vg_replace_malloc.c:538)
==20==    by 0x48E2134: strerror_l (in /usr/lib64/libc-2.32.so)
==20==    by 0x109328: main (test-perror2.c:72)
==20==  Block was alloc'd at
==20==    at 0x4839809: malloc (vg_replace_malloc.c:307)
==20==    by 0x48CA03F: __vasprintf_internal (in /usr/lib64/libc-2.32.so)
==20==    by 0x48A46F9: asprintf (in /usr/lib64/libc-2.32.so)
==20==    by 0x48E2184: strerror_l (in /usr/lib64/libc-2.32.so)
==20==    by 0x1092E2: main (test-perror2.c:67)

Likewise the test-strerror_r failure:

==21== Invalid read of size 1
==21==    at 0x483DAB4: strcmp (vg_replace_strmem.c:847)
==21==    by 0x109660: main (test-strerror_r.c:170)
==21==  Address 0x4a1a1b0 is 0 bytes inside a block of size 17 free'd
==21==    at 0x483A9F5: free (vg_replace_malloc.c:538)
==21==    by 0x48E2134: strerror_l (in /usr/lib64/libc-2.32.so)
==21==    by 0x1095B7: main (test-strerror_r.c:161)
==21==  Block was alloc'd at
==21==    at 0x4839809: malloc (vg_replace_malloc.c:307)
==21==    by 0x48CA03F: __vasprintf_internal (in /usr/lib64/libc-2.32.so)
==21==    by 0x48A46F9: asprintf (in /usr/lib64/libc-2.32.so)
==21==    by 0x48E2184: strerror_l (in /usr/lib64/libc-2.32.so)
==21==    by 0x10958B: main (test-strerror_r.c:156)
==21== 
==21== Invalid read of size 1
==21==    at 0x483DAC8: strcmp (vg_replace_strmem.c:847)
==21==    by 0x109660: main (test-strerror_r.c:170)
==21==  Address 0x4a1a1b1 is 1 bytes inside a block of size 17 free'd
==21==    at 0x483A9F5: free (vg_replace_malloc.c:538)
==21==    by 0x48E2134: strerror_l (in /usr/lib64/libc-2.32.so)
==21==    by 0x1095B7: main (test-strerror_r.c:161)
==21==  Block was alloc'd at
==21==    at 0x4839809: malloc (vg_replace_malloc.c:307)
==21==    by 0x48CA03F: __vasprintf_internal (in /usr/lib64/libc-2.32.so)
==21==    by 0x48A46F9: asprintf (in /usr/lib64/libc-2.32.so)
==21==    by 0x48E2184: strerror_l (in /usr/lib64/libc-2.32.so)
==21==    by 0x10958B: main (test-strerror_r.c:156)

Both are masked by malloc behavior with a 16-byte heap alignment (only armhfp has 8-byte heap alignment). I reported it to gnulib here: https://lists.gnu.org/archive/html/bug-gnulib/2020-08/msg00220.html

I believe Jakub has fixed the test-float issue on POWER in upstream GCC: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95450

Comment 7 Jaroslav Škarvada 2020-08-27 07:54:10 UTC
Thanks Florian, good work. It was on my todo list, but unfortunately I wasn't able to get to it :)

Comment 8 Jeff Law 2020-08-27 15:48:22 UTC
The fix for the test-float issues on ppc64 are supposed to be in the next gcc build according to a recent discussion between Jakub and myself.

Comment 9 Adam Williamson 2020-08-27 17:37:09 UTC
Thanks! So if I'm getting this right, the test-float bug is a genuine toolchain (gcc) issue, so we should wait for the gcc fix before rebuilding grep. It looks like the fix is in gcc-10.2.1-3 which was built for Rawhide yesterday and is building for F33 right now?

The other two bugs are in the tests themselves, so it would be OK to just disable those two tests on armv7hl to get the build through, if no-one provides a fix for the tests in time (I'm not gonna attempt that with my C skills :P) Right? Thanks again.

Comment 10 Jeff Law 2020-08-27 17:43:24 UTC
Yes, the test-float is a gcc bug.  I believe the gcc build is waiting on s390x (big surprise there).

For the others I'd disable LTO for the entire build on the affected architecture.
%ifarch armv7hl
%define _lto_cflags %{nil}
%endif

That style is caught by my .spec file scanner that searches for LTO opt-outs that need deeper investigation.

Comment 11 Adam Williamson 2020-08-27 18:01:02 UTC
Awesome. Thanks. Will do that.

Comment 12 Adam Williamson 2020-08-28 17:38:08 UTC
Build fails even with that :/ Not sure if it's because LTO isn't actually the problem, or if the spec using $RPM_OPT_FLAGS the way it does ignores the change to _lto_cflags.

https://koji.fedoraproject.org/koji/taskinfo?taskID=50339572

Comment 13 Adam Williamson 2020-08-28 17:42:17 UTC
The build log doesn't contain `-flto=auto`, so I guess disabling LTO did work, but doesn't fix the tests.

Comment 14 Fedora Update System 2020-08-28 18:09:03 UTC
FEDORA-2020-bc38b42372 has been pushed to the Fedora 34 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 15 Adam Williamson 2020-08-28 18:29:10 UTC
OK, I just used the patch from Paul Eggert to remove the offending checks, in the end, and got a build through for Rawhide. F33 is still waiting on gcc to finish building on s390 :(

Comment 16 Fedora Update System 2020-08-28 21:36:20 UTC
FEDORA-2020-81fea835a4 has been submitted as an update to Fedora 33. https://bodhi.fedoraproject.org/updates/FEDORA-2020-81fea835a4

Comment 17 Fedora Update System 2020-08-31 14:27:33 UTC
FEDORA-2020-81fea835a4 has been pushed to the Fedora 33 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-81fea835a4`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-81fea835a4

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 18 Fedora Update System 2020-09-02 19:48:40 UTC
FEDORA-2020-81fea835a4 has been pushed to the Fedora 33 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 19 Fedora Update System 2020-09-15 20:48:31 UTC
FEDORA-2020-0dcd106c75 has been pushed to the Fedora ELN stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.