Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1352625 - Duplicate __syscall_error in librt.a and libpthread.a causing static linking to fail
Summary: Duplicate __syscall_error in librt.a and libpthread.a causing static linking ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: glibc
Version: 24
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
Assignee: Florian Weimer
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1356523 (view as bug list)
Depends On:
Blocks: ZedoraTracker
TreeView+ depends on / blocked
 
Reported: 2016-07-04 12:57 UTC by Daniel Berrangé
Modified: 2016-09-05 08:40 UTC (History)
13 users (show)

Fixed In Version: glibc-2.22-18.fc23, glibc-2.23.1-10.fc24, glibc-2.24-3.fc25, glibc-2.24.90-3.fc26
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-09-05 08:40:03 UTC
Type: Bug
Embargoed:

Attachments (Terms of Use)
Don't force link of sysdep.o in libpthread.a (835 bytes, patch)
2016-07-04 17:18 UTC, Daniel Berrangé 		no flags Details | Diff
qemu build fix for ppc64(le) (871 bytes, patch)
2016-08-17 07:08 UTC, Sinny Kumari 		no flags Details | Diff
View All


Links
System ID Private Priority Status Summary Last Updated
Sourceware 20452 0 P2 RESOLVED Addition of sysdep.o to libpthread.a breaks relinking libpthread.a 2021-01-26 14:59:11 UTC

Description Daniel Berrangé 2016-07-04 12:57:16 UTC 
Description of problem:
Try to build a static binary that links to librt.a and libpthread.a will fail due to duplicate symbols. 

I've tested armv7, x86_64 and i686 and this only happens on i686.

This is seen when trying to build qemu user emulators as static libs. eg this failure

https://kojipkgs.fedoraproject.org//work/tasks/2427/14732427/build.log

It is easily reproduced standalone though with this program:

$ cat > foo.c <<EOF
#include <stdio.h>
#include <mqueue.h>
#include <pthread.h>

int main(void)
{

    pthread_mutex_t lock;
    pthread_mutex_lock(&lock);
    mq_open("foo", O_WRONLY);
    fprintf(stderr, "Hello World\n");
}
EOF

$ gcc -static -o foo foo.c -pthread  -lrt
/usr/lib/gcc/i686-redhat-linux/6.1.1/../../../libpthread.a(libpthread.o): In function `__syscall_error':
(.text+0xb9b0): multiple definition of `__syscall_error'
/usr/lib/gcc/i686-redhat-linux/6.1.1/../../../librt.a(sysdep.o):(.text+0x0): first defined here
collect2: error: ld returned 1 exit status

It is broken in Fedora 24 and rawhide

glibc-static-2.23.1-8.fc24.i686
glibc-static-2.23.90-25.fc25.i686

It works fine in Fedora 23:

glibc-static-2.22-17.fc23.i686
Comment 1 Daniel Berrangé 2016-07-04 13:03:24 UTC 
Looking at glibc GIT history upstream, the following commit looks like a candidate for causing the regression

commit fb1cf108115f7e2e4510859693431b3473657d77
Author: H.J. Lu <hjl.tools>
Date:   Tue Oct 13 11:58:53 2015 -0700

    Add INLINE_SYSCALL_ERROR_RETURN_VALUE
    
    For ia32 PIC, the first thing of many syscalls does is to call
    __x86.get_pc_thunk.reg to load PC into reg in case there is an error,
    which is required for setting errno.  In most cases, there are no
    errors.  But we still call __x86.get_pc_thunk.reg.  This patch adds
    INLINE_SYSCALL_ERROR_RETURN_VALUE so that i386 can optimize setting
    errno by branching to the internal __syscall_error without PLT.
    
    With i386 INLINE_SYSCALL_ERROR_RETURN_VALUE and i386 syscall inlining
    optimization for GCC 5, for sysdeps/unix/sysv/linux/fchmodat.c with
    -O2 -march=i686 -mtune=generic, GCC 5.2 now generates:
    

It was added in 2.23, and this particular diff looks like it is pulling the __syscall_error function into librt and libpthread

diff --git a/sysdeps/unix/sysv/linux/i386/Makefile b/sysdeps/unix/sysv/linux/i386/Makefile
index 80da593..e10d133 100644
--- a/sysdeps/unix/sysv/linux/i386/Makefile
+++ b/sysdeps/unix/sysv/linux/i386/Makefile
@@ -27,3 +27,17 @@ endif
 ifeq ($(subdir),stdlib)
 gen-as-const-headers += ucontext_i.sym
 endif
+
+ifeq ($(subdir),csu)
+sysdep-dl-routines += sysdep
+endif
+
+ifeq ($(subdir),nptl)
+# pull in __syscall_error routine
+libpthread-routines += sysdep
+endif
+
+ifeq ($(subdir),rt)
+# pull in __syscall_error routine
+librt-routines += sysdep
+endif
Comment 2 Florian Weimer 2016-07-04 13:13:55 UTC 
I suspect it's not an upstream bug because upstream does not link libpthread.a into a single object file, libpthread.o.
Comment 3 Daniel Berrangé 2016-07-04 13:29:22 UTC 
Yep, looks like you are right. I built latest upstream glibc and it worked fine. I then copied the fedora glibc.spec hac

gcc -r -nostdlib -o libpthread.o -Wl,--whole-archive ./libpthread.a
ar rcs libpthread.a libpthread.o

and it now shows the problem
Comment 4 Daniel Berrangé 2016-07-04 17:18:58 UTC 
Created attachment 1176144 [details]
Don't force link of sysdep.o in libpthread.a

Personally I'd just vote for removing the Fedora specific hack, so libpthread.a in Fedora behaves the same way as on all other Linux distros.

If we can't do that though, then this patch to the specfile tries to workaround the problem, by excluding sysdep.o from the objects we merge into libpthread.o.
Comment 5 Daniel Berrangé 2016-07-14 10:19:03 UTC 
*** Bug 1356523 has been marked as a duplicate of this bug. ***
Comment 6 Daniel Berrangé 2016-07-14 10:20:39 UTC 
NB, also affects s390 platform
Comment 7 Dan Horák 2016-07-14 10:34:33 UTC 
(In reply to Daniel Berrange from comment #6)
> NB, also affects s390 platform

let's check also ppc64/ppc64le with the standalone test case ...
Comment 8 Dan Horák 2016-07-14 10:40:35 UTC 
(In reply to Dan Horák from comment #7)
> (In reply to Daniel Berrange from comment #6)
> > NB, also affects s390 platform
> 
> let's check also ppc64/ppc64le with the standalone test case ...

no problem seen there
Comment 9 Carlos O'Donell 2016-07-16 03:29:30 UTC 
We can't remove the libpthread.a relinking without breaking other use cases.

Thus I think splitting out sysdep.o from the other object files is a sensible solution, but I'd need to verify the final symbols that are moved out.
Comment 10 Dan Horák 2016-07-26 09:29:26 UTC 
Hi guys, is there any progress with this? The missing qemu build failing due this issue is blocking rawhide builds on s390x.
Comment 11 Dan Horák 2016-07-26 13:04:38 UTC 
next qemu build (qemu-2.6.0-6.fc25) was successful, so this is not blocking anymore
Comment 12 Daniel Berrangé 2016-07-26 13:06:05 UTC 
(In reply to Dan Horák from comment #11)
> next qemu build (qemu-2.6.0-6.fc25) was successful, so this is not blocking
> anymore

That's only because i temporarily disabled static qemu-user on s390(x) in qemu RPM spec until this glibc bug is fixed
Comment 13 Florian Weimer 2016-08-04 09:20:24 UTC 
Patch posted upstream: https://sourceware.org/ml/libc-alpha/2016-08/msg00180.html
Comment 14 Florian Weimer 2016-08-04 11:34:30 UTC 
Upstream branches for 2.23, 2.24 and master now contain this fix.
Comment 15 Dan Horák 2016-08-11 10:33:57 UTC 
Hmm, seems the test case is passing on ppc64 and ppc64le, but qemu still fails (at least on ppc64le) with

...
cc -I/builddir/build/BUILD/qemu-2.6.0/dtc/libfdt -m64 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common  -O2 -g -pipe -Wall -Werror=format-security  -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mcpu=power8 -mtune=power8 -Wendif-labels -Wmissing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-strong -I/usr/include/cacard -I/usr/include/nss3 -I/usr/include/nspr4 -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/libusb-1.0   -I../linux-headers -I.. -I/builddir/build/BUILD/qemu-2.6.0/target-arm -DNEED_CPU_H -I/builddir/build/BUILD/qemu-2.6.0/include -I/builddir/build/BUILD/qemu-2.6.0/linux-user/aarch64 -I/builddir/build/BUILD/qemu-2.6.0/linux-user -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -g  -Wl,--warn-common -m64 -static -g  -Wl,--build-id -Wl,-z,relro -Wl,-z,now -Wl,-Ttext-segment=0x60000000 -o qemu-aarch64 exec.o translate-all.o cpu-exec.o translate-common.o cpu-exec-common.o tcg/tcg.o tcg/tcg-op.o tcg/optimize.o tcg/tcg-common.o fpu/softfloat.o disas.o gdbstub-xml.o kvm-stub.o gdbstub.o thunk.o user-exec.o linux-user/main.o linux-user/syscall.o linux-user/strace.o linux-user/mmap.o linux-user/signal.o linux-user/elfload.o linux-user/linuxload.o linux-user/uaccess.o linux-user/uname.o linux-user/flatload.o target-arm/arm-semi.o target-arm/kvm-stub.o target-arm/translate.o target-arm/op_helper.o target-arm/helper.o target-arm/cpu.o target-arm/neon_helper.o target-arm/iwmmxt_helper.o target-arm/gdbstub.o target-arm/cpu64.o target-arm/translate-a64.o target-arm/helper-a64.o target-arm/gdbstub64.o target-arm/crypto_helper.o ../tcg-runtime.o ../disas/arm.o ../disas/ppc.o ../hw/core/qdev.o ../hw/core/qdev-properties.o ../hw/core/fw-path-provider.o ../hw/core/irq.o ../hw/core/hotplug.o ../hw/core/nmi.o ../qom/cpu.o trace/generated-helpers.o ../qom/object.o ../qom/container.o ../qom/qom-qobject.o ../qom/object_interfaces.o ../crypto/aes.o  ../libqemuutil.a ../libqemustub.a   -lm -lgthread-2.0 -pthread -lglib-2.0 -pthread -lpcre -pthread  -lz -lrt
/usr/lib/gcc/ppc64le-redhat-linux/6.1.1/../../../../lib64/libpthread.a(libpthread.o): In function `__syscall_error':
(.text+0x13170): multiple definition of `__syscall_error'
/usr/lib/gcc/ppc64le-redhat-linux/6.1.1/../../../../lib64/librt.a(rt-sysdep.o):(.text+0x0): first defined here
/usr/lib/gcc/ppc64le-redhat-linux/6.1.1/../../../../lib64/libglib-2.0.a(libglib_2_0_la-gutils.o): In function `g_get_user_database_entry':
(.text+0x2d0): warning: Using 'getpwuid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/ppc64le-redhat-linux/6.1.1/../../../../lib64/libglib-2.0.a(libglib_2_0_la-gutils.o): In function `g_get_user_database_entry':
(.text+0x178): warning: Using 'getpwnam_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/ppc64le-redhat-linux/6.1.1/../../../../lib64/libglib-2.0.a(libglib_2_0_la-gutils.o): In function `g_get_user_database_entry':
(.text+0x1bc): warning: Using 'getpwuid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/ppc64le-redhat-linux/6.1.1/../../../../lib64/libc.a(dl-sysdep.o): warning: multiple common of `__cache_line_size'
/usr/lib/gcc/ppc64le-redhat-linux/6.1.1/../../../../lib64/libc.a(libc-start.o): warning: previous common is here
collect2: error: ld returned 1 exit status
Makefile:195: recipe for target 'qemu-aarch64' failed
make[1]: Leaving directory '/builddir/build/BUILD/qemu-2.6.0/build-static/aarch64-linux-user'
make[1]: *** [qemu-aarch64] Error 1

(http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3591086)
Comment 16 Dan Horák 2016-08-11 10:36:56 UTC 
fails on both, http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3591555
Comment 17 Daniel Berrangé 2016-08-11 10:57:17 UTC 
(In reply to Dan Horák from comment #15)
> Hmm, seems the test case is passing on ppc64 and ppc64le, but qemu still
> fails (at least on ppc64le) with

[snip]

> /usr/lib/gcc/ppc64le-redhat-linux/6.1.1/../../../../lib64/libc.a(dl-sysdep.o): warning: multiple common of `__cache_line_size'
> /usr/lib/gcc/ppc64le-redhat-linux/6.1.1/../../../../lib64/libc.a(libc-start.o): warning: previous common is here
> collect2: error: ld returned 1 exit status

NB, that's a different failure that the original one QEMU hit which was a clash between librt.a vs libpthread.a
Comment 18 Florian Weimer 2016-08-11 10:57:51 UTC 
(In reply to Dan Horák from comment #16)
> fails on both, http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3591555

I don't think we have cut a Fedora release after the upstream changes.
Comment 19 Florian Weimer 2016-08-11 12:15:56 UTC 
It turns out that POWER needs a separate fix (also missing upstream).
Comment 20 Dan Horák 2016-08-11 12:21:03 UTC 
(In reply to Daniel Berrange from comment #17)
> (In reply to Dan Horák from comment #15)
> > Hmm, seems the test case is passing on ppc64 and ppc64le, but qemu still
> > fails (at least on ppc64le) with
> 
> [snip]
> 
> > /usr/lib/gcc/ppc64le-redhat-linux/6.1.1/../../../../lib64/libc.a(dl-sysdep.o): warning: multiple common of `__cache_line_size'
> > /usr/lib/gcc/ppc64le-redhat-linux/6.1.1/../../../../lib64/libc.a(libc-start.o): warning: previous common is here
> > collect2: error: ld returned 1 exit status
> 
> NB, that's a different failure that the original one QEMU hit which was a
> clash between librt.a vs libpthread.a

yes, this is new, not sure whether this is fatal (is marked as warning), but you can see the __syscall_error there too
Comment 21 Sinny Kumari 2016-08-17 07:08:48 UTC 
Created attachment 1191480 [details]
qemu build fix for ppc64(le)

Until qemu fix for POWER is done, I am proposing to disable qemu-user-static sub-RPM for power arches too. I have prepared a patch(available in bugzilla attachment) which works fine for ppc64(le) build and shouldn't affect other arches.

koji scratch build - http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3612293 (F25) and http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3612300 (rawhide)
Comment 22 Sinny Kumari 2016-08-18 06:31:16 UTC 
With glibc-2.24.90-3.fc26, qemu builds fine on Fedora powerpc, rawhide.
Scratch build link - http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3615007 .

It will be great if patch can be backported to F25 and F24
Comment 23 Fedora Update System 2016-08-18 14:55:58 UTC 
glibc-2.24-3.fc25 has been submitted as an update to Fedora 25. https://bodhi.fedoraproject.org/updates/FEDORA-2016-57cba655d5
Comment 24 Fedora Update System 2016-08-18 15:17:17 UTC 
glibc-2.23.1-10.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2016-5f050a0a6d
Comment 25 Fedora Update System 2016-08-18 16:53:31 UTC 
glibc-2.24-3.fc25 has been pushed to the Fedora 25 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-57cba655d5
Comment 26 Fedora Update System 2016-08-18 17:33:04 UTC 
glibc-2.22-18.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2016-87dde780b8
Comment 27 Fedora Update System 2016-08-19 00:22:05 UTC 
glibc-2.22-18.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-87dde780b8
Comment 28 Fedora Update System 2016-08-19 00:58:42 UTC 
glibc-2.23.1-10.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-5f050a0a6d
Comment 29 Fedora Update System 2016-08-19 19:52:41 UTC 
glibc-2.23.1-10.fc24 has been pushed to the Fedora 24 stable repository. If problems still persist, please make note of it in this bug report.
Comment 30 Fedora Update System 2016-08-27 10:22:32 UTC 
glibc-2.24-3.fc25 has been pushed to the Fedora 25 stable repository. If problems still persist, please make note of it in this bug report.
Comment 31 Fedora Update System 2016-09-02 23:20:59 UTC 
glibc-2.22-18.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.
Note You need to log in before you can comment on or make changes to this bug.