Sorry, dpdk-17.11-4.fc28.x86_64 regressed.  BIND_NOW (aka -Wl,-z,now) is now missing.  I suspect it's because you removed “-e's/,/ /g'” from the sed expression.

Why can't you use the build flags directly?

have you looked at the DPDK build system?  Its a hot mess.  It mangles LDFLAGS, and EXTRA_LDFLAGS to insert -Wl automagically, which is already contained in the rpm macros, so they can't be used directly without further mangling them (hence the sed magic)

As far as your assertion of regression goes, I'm not sure I follow you.  I see the use -z now in the build logs:


https://koji.fedoraproject.org/koji/taskinfo?taskID=25373883

I think the problem was that I accidentally re-ran a scratch build after I added that last commit, which fixes the problem you describe, rather than creates it.

Can we merge the latest upstream version into Fedora 28?  I would like to keep Fedora 28 as polished as possible, from a build flags point of view.

18.02 is the latest version.  I merged that last week (on Feb 19th).

(In reply to Neil Horman from comment #4)
> 18.02 is the latest version.  I merged that last week (on Feb 19th).

That was rawhide.  I'm interested in a fix for Fedora 28.

oh, if we've already forked, then yeah, we can update there as well.

actually, scratch that, I don't see an f28 branch for anything yet.  Shouldn't we still be inheriting from rawhide?

(In reply to Neil Horman from comment #7)
> actually, scratch that, I don't see an f28 branch for anything yet. 
> Shouldn't we still be inheriting from rawhide?

We have already branched, and fixed dpdk packages have not yet been built for Fedora 28.

[nhorman@hmswarspite dpdk]$ fedpkg request-branch f28
Could not execute request_branch: f28 is not a current release branch


Seems like something isn't quite right then.  Any ideas?

I don't understand what you are trying to do.  There is already an f28 branch:

https://src.fedoraproject.org/rpms/dpdk/commits/f28

Thats very strange, it wasn't showing up in my git tree even after a pull, but deleting and recreating my clone caused it to show up.  Ok, I'll update f28

Sorry, but it turns out that injection is still missing for dpdk-pmdinfogen.  From the dpdk-devel-18.02-2.fc28.x86_64 build log:

gcc -Wp,-MD,./.pmdinfogen.o.d.tmp  -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wdeprecated -Wimplicit-fallthrough=2 -Wno-format-truncation -g -I/builddir/build/BUILD/dpdk-18.02/x86_64-native-linuxapp-gcc/include    -o pmdinfogen.o -c /builddir/build/BUILD/dpdk-18.02/buildtools/pmdinfogen/pmdinfogen.c 
gcc    -o dpdk-pmdinfogen pmdinfogen.o  

But on the positive side, this is the only problematic file which remains on x86-64 according to the current tests.

Ugh, apparently binaries build as part of the hostapp class in the dpdk buildsystem use a different linker macro, so it didn't pick up the global one I changed before.  I'll fix it

Thanks.  Would you please put the fix into Fedora 28 as well?

dpdk-18.02-4.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2018-38209f4385

dpdk-18.02-4.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-38209f4385

Sorry, the build flags are still not correct in dpdk-devel-18.02-4.fc28.x86_64:

gcc   -z,relro -z,now  -o dpdk-pmdinfogen pmdinfogen.o  
/usr/bin/ld: warning: -z ,relro ignored.
/usr/bin/ld: warning: -z ,now ignored.

So /usr/share/dpdk/x86_64-default-linuxapp-gcc/app/dpdk-pmdinfogen is still not linked correctly.

do me a favor, open a new bug for this, the flag are properly set, but something in the DPDK makefile is broken.  This will need to be fixed up stream as well

actually, nm, it still needs fixing upstream, but I may be able to work around it in the spec file

dpdk-18.02-6.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2018-0c67c5fdc3

dpdk-18.02-6.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-0c67c5fdc3

Sorry, /usr/share/dpdk/x86_64-default-linuxapp-gcc/app/dpdk-pmdinfogen in dpdk-devel-18.02-6.fc28.x86_64 is still not PIE. 8-(

gcc   -Wl,-z,relro -Wl,-z,now  -o dpdk-pmdinfogen pmdinfogen.o  

You really need the full %{build_ldflags} there.

I did a scratch build:
https://koji.fedoraproject.org/koji/taskinfo?taskID=26270085

In which I replaced __global_ldflags with build_ldflags, and no additional -fPIE flags were observed.

Looking at my fedora rpm macros on my local system, it appears that -fPIE isn't set in any of the ldflags macros.

Are you sure its intended to be there?

(In reply to Neil Horman from comment #23)
> Looking at my fedora rpm macros on my local system, it appears that -fPIE
> isn't set in any of the ldflags macros.

We have two goals here: Use the Fedora flags so that future changes are reflected automatically in the package builds, and produce the intended hardening settings (which includes PIE).

The mechanism to enable PIE is somewhat complicated, search for “-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1” here:

https://src.fedoraproject.org/rpms/redhat-rpm-config/blob/master/f/buildflags.md

So you won't necessarily see -fPIE on linker command lines.  This is fine, as long as -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 is there.

Ok, I see what the problem is.  The -spec line was getting removed because its neither a linker nor a gcc flag, but rather some spec file macro expansion magic (for which I can find no documentation).  I can't just use __hardened_ldflags as is because dpdk requires that these flags get massaged into their non-standard build flags.  I'm currently trying to use the %set_build_flags macro to expand CFLAGS and LDFLAGS and use those, but it just expands %{_hardened_ldflags} to    
 -specs=/usr/lib/rpm/redhat/hardened-redhat-ld, which the linker doesn't recognize.  Whats the proscribed method for expanding that argument to the expanded contents of /usr/lib/rpm/redhat/redhat-hardened-ld?

(In reply to Neil Horman from comment #25)
> Ok, I see what the problem is.  The -spec line was getting removed because
> its neither a linker nor a gcc flag, but rather some spec file macro
> expansion magic (for which I can find no documentation).  I can't just use
> __hardened_ldflags as is because dpdk requires that these flags get massaged
> into their non-standard build flags.  I'm currently trying to use the
> %set_build_flags macro to expand CFLAGS and LDFLAGS and use those, but it
> just expands %{_hardened_ldflags} to    
>  -specs=/usr/lib/rpm/redhat/hardened-redhat-ld, which the linker doesn't
> recognize.  Whats the proscribed method for expanding that argument to the
> expanded contents of /usr/lib/rpm/redhat/redhat-hardened-ld?

The recommended way is to invoke the linker through the gcc compiler driver.  The requirements for PIE, static linking, shared objects etc. are otherwise too hard to express with a single set of command line flags.

I'm asking around if there is a way to run the gcc compiler driver and have it print the linker command line, without actually running the linker, but I don't think there is.

Jakub Jelinek pointed me to the -### (sic) option.

You can do this:

touch obj.o
gcc -### obj.o
gcc -### $(rpm --eval '%{build_ldflags}') obj.o

And then compare the difference in the set of flags on the lines matching /^ *[^ ]*collect2/.

Ok, I think I've managed to get something cobbled together (I hate the DPDK build system).  Do me a favor and take a look at the build logs here please:
https://koji.fedoraproject.org/koji/taskinfo?taskID=28603326

And confirm that you see all the c/ldflags that you expect.

(In reply to Neil Horman from comment #28)
> Ok, I think I've managed to get something cobbled together (I hate the DPDK
> build system).  Do me a favor and take a look at the build logs here please:
> https://koji.fedoraproject.org/koji/taskinfo?taskID=28603326
> 
> And confirm that you see all the c/ldflags that you expect.

Sorry, the scratch build has expired by now.  Would you mind creating another one with your changes (for rawhide)?  Thanks.

https://koji.fedoraproject.org/koji/taskinfo?taskID=29862773

(In reply to Neil Horman from comment #30)
> https://koji.fedoraproject.org/koji/taskinfo?taskID=29862773

I still see these failures:

Hardened: dpdk-procinfo: MAYB: The PIC/PIE setting was not recorded.
Hardened: dpdk-procinfo: FAIL: Not linked as a position independent executable (ie need to add '-pie' to link command line).
Hardened: testpmd: MAYB: The PIC/PIE setting was not recorded.
Hardened: testpmd: FAIL: Not linked as a position independent executable (ie need to add '-pie' to link command line).

There is also this on x86_64:

Hardened: librte_pmd_af_packet.so.1: FAIL: GNU Property note is missing, but -fcf-protection is enabled.

That could be due to some included assembler routines which lack CET markup.

It does not happen on aarch64.  (On ppc64le, the annobin tooling is currently broken.)

https://koji.fedoraproject.org/koji/taskinfo?taskID=29867672

Try that, turns out I can't just use -pie as an LDFLAG, as it breaks some of the shared library builds

not sure whats going on with the fcf-protection issue

(In reply to Neil Horman from comment #32)
> https://koji.fedoraproject.org/koji/taskinfo?taskID=29867672
> 
> Try that, turns out I can't just use -pie as an LDFLAG, as it breaks some of
> the shared library builds

You can compile everything with -fPIC, that's not a problem, it's just a minor loss in performance on some targets.  More recent versions of the annocheck should not report the use of PIC with executables anymore.

But you need to link executables (and only executables) with -pie.  If you use %{build_ldflags} for linking, it will automatically do the right thing: PIE link by default, shared object link (without PIE) with -shared.

> not sure whats going on with the fcf-protection issue

Do you have assembler code?  Upstream will have to fix that.

I know what I need to do, the problem is that DPDK builds both libraries and executables with the same flags that get passed in from the spec file.

And yes, theres lots of assembly in dpdk, and its there intentionally, so I'm not sure what you mean by "upstream will have to fix that".  They're not in the habbit of changing anything that might impact performance.

whats  the exact annocheck command line your using to validate this.  I'll run it here to see if I'm missing anything else

(In reply to Neil Horman from comment #34)
> I know what I need to do, the problem is that DPDK builds both libraries and
> executables with the same flags that get passed in from the spec file.

It absolutely has to use -shared for linking libraries, and our %{build_flags} flags collection has magic in it which will disable the PIE link with -shared.  This is why we have to use a -specs= option as part of %{build_flags}.

So you absolutely can use the same build flags for libraries and executables.  Your build system is not the only one having this problem.

> And yes, theres lots of assembly in dpdk, and its there intentionally, so
> I'm not sure what you mean by "upstream will have to fix that".  They're not
> in the habbit of changing anything that might impact performance.

That's why the assembler files need review.  #include <cet.h> by itself does not add any new code, but depending on what the assembler files contain and how the functions therein are called, it maybe necessary to add CET markup instructions (which current CPUs treat as a NOP).

> whats  the exact annocheck command line your using to validate this.  I'll
> run it here to see if I'm missing anything else

I personally use “annocheck -v”.  The various --skip flags aren't necessary if you just pass through the Fedora build flags.

dpdk-17.11.2-2.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2018-3350c45b17

dpdk-17.11.2-2.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-3350c45b17

This message is a reminder that Fedora 28 is nearing its end of life.
On 2019-May-28 Fedora will stop maintaining and issuing updates for
Fedora 28. It is Fedora's policy to close all bug reports from releases
that are no longer maintained. At that time this bug will be closed as
EOL if it remains open with a Fedora 'version' of '28'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 28 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Fedora 28 changed to end-of-life (EOL) status on 2019-05-28. Fedora 28 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.