I complied and ran test locally on a ppc64 box to find reason of test failure. So far I found following minute details which may be useful:
Comment lines 857 and 858 of source file https://github.com/cockpit-project/cockpit/blob/0.79/src/ws/test-sshtransport.c to avoid running sub-tests key-auth-message and key-auth-failed .

After commenting them, all tests passes and F22 sub-packages build successfully too.

I will look further and will update reason of these tests failure.

also worth mentioning that only big endian arches seems affected, ppc64 and s390/s390x

Tracked this down to a bug with libssh. While we are waiting for a fix do we want to disable key auth in cockpit on these arches. Or are we ok holding up these builds till it gets fixed.

(In reply to Peter from comment #5)
> Tracked this down to a bug with libssh. While we are waiting for a fix do we
> want to disable key auth in cockpit on these arches. Or are we ok holding up
> these builds till it gets fixed.

We would prefer a new cockpit release (can be just patched Fedora package) until there is fixed libssh (is the issue already tracked in separate bugzilla?). We don't have a new cockpit build for quite some time and cockpit is part of the Fedora Server flavour we produce for ppc64 and s390x.

Reassigning to libssh. Peter do you have more details of the bug?

upstream bug report. https://red.libssh.org/issues/204

I applied provided patch for libssh (tried on both 0.7.1 and 0.7.2) https://git.libssh.org/users/asn/libssh.git/commit/?h=fix&id=dbe409ccdd59e25f5ea3639d5f4f91ed3c570a36 (additionally modifying #define bswap_32 to #define bswap_32(x) to avoid compilation error) and installed locally on ppc64 box.

Further tried building again cockpit-0.79, but I still see test failing:
FAIL: test-sshtransport 8 /ssh-transport/key-auth-message

For reference, latest build.log file is available at https://sinnykumari.fedorapeople.org/cockpit/build.log

Just to be clear, the patch upstream does not fix the issue. I know it's confusing because the error message contains the word "Success" but it's still a failure.

With the latest libssh patch https://git.libssh.org/users/asn/libssh.git/commit/?h=fix&id=d60cfa24861394b1f5a8d80e26dadb41b3d3829e , cockpit-0.79 build failure has been resolved.

Successful local build log of cockpit-0.79 is available at https://sinnykumari.fedorapeople.org/cockpit/fixed_build/build.log .

I think we can now build cockpit-0.79 on big endian with newer libssh build(having "fix agent auth on big endian machines" patch included).

libssh-0.7.2-2.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2015-66ba300501

libssh-0.7.2-2.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2015-b242195d7f

libssh-0.7.2-2.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'dnf --enablerepo=updates-testing update libssh'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-66ba300501

Proposed as a Blocker and Freeze Exception for 23-final by Fedora user karsten using the blocker tracking app because:

 This is blocking cockpit builds on bigendian secondary archs.
Requesting freeze exception as cockpit is required for the Server edition.

*** Bug 1275229 has been marked as a duplicate of this bug. ***

Proposed as a Freeze Exception for 23-beta by Fedora user pbrobinson using the blocker tracking app because:

 The libssh breaks builds of cockpit on big endian architectures where cockpit is a core feature of the Server spins we ship. 

Proposing as a exception as it's for secondary arches

Discussed at 2015-10-26 blocker review meeting: [1]. 

This bug was accepted as Final blocker: This bug doesn't qualify for blocker status, but we'd consider pulling in a tested fix despite freeze.

[1] https://meetbot.fedoraproject.org/fedora-blocker-review/2015-10-26/f23-blocker-review.2015-10-26-16.00.html

Sorry for the confusion. I have really a bad day. This bug was rejected as blocker but accepted as Freeze Exception.

libssh-0.7.2-2.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'dnf --enablerepo=updates-testing update libssh'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-b242195d7f

For the record, I'm leaving this out of the RC4 compose, as it's big-endian only there's no point pulling it into the compose for primary arches, all of which are little-endian. The point of the FE is to get it pushed stable, and that will happen when it has sufficient karma.

(In reply to awilliam from comment #21)
> For the record, I'm leaving this out of the RC4 compose, as it's big-endian
> only there's no point pulling it into the compose for primary arches, all of
> which are little-endian. The point of the FE is to get it pushed stable, and
> that will happen when it has sufficient karma.

We need it stable for the GA f23 tag

Isn't libssh on the Server DVD? I don't think it can go stable if it wasn't part of the compose because we have to make sure that the version that's on the images matches with the version that's in the stable repo.

Oh, yes, sorry - you're right. I missed that, was trying to avoid changing anything potentially risky for RC4. Well, if we get to RC5 we'll pull it in. my bad!

libssh-0.7.2-2.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.

libssh-0.7.2-2.fc22 has been pushed to the Fedora 22 stable repository. If problems still persist, please make note of it in this bug report.