Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 2121490 - Review Request: rust-sodiumoxide - Fast cryptographic library for Rust
Summary: Review Request: rust-sodiumoxide - Fast cryptographic library for Rust
Keywords:
Status: ASSIGNED
Alias: None
Product: Fedora
Classification: Fedora
Component: Package Review
Version: rawhide
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Fabio Valentini
QA Contact: Fedora Extras Quality Assurance
URL: https://crates.io/crates/sodiumoxide
Whiteboard:
Depends On: rust-ed25519 1991164 2117954 2193407
Blocks: 2045255
TreeView+ depends on / blocked
 
Reported: 2022-08-25 15:46 UTC by Stuart D Gathman
Modified: 2023-12-09 23:12 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Type: ---
Embargoed:
decathorpe: fedora-review?


Attachments (Terms of Use)

Description Stuart D Gathman 2022-08-25 15:46:48 UTC
Spec URL: https://gathman.org/linux/SPECS/rust-sodiumoxide.spec
SRPM URL: https://gathman.org/linux/f37/src/rust-sodiumoxide-0.2.7-1.fc37.src.rpm
Description: Fast cryptographic library for Rust (bindings to libsodium).
Fedora Account System Username: sdgathman

While this was built on by dev system, it is still waiting for dependencies.  I am adding the review request now to document the holdup for Cjdns.

Comment 1 Stuart D Gathman 2022-10-13 00:14:14 UTC
Spec URL: https://gathman.org/linux/SPECS/rust-sodiumoxide.spec
SRPM URL: https://gathman.org/linux/f37/src/rust-sodiumoxide-0.2.7-2.fc37.src.rpm

One of the tests fails - a missing compile dependency:

failures:
    src/newtype_macros.rs - newtype_macros::new_type (line 175)
    src/newtype_macros.rs - newtype_macros::new_type (line 184)
    src/newtype_macros.rs - newtype_macros::new_type (line 194)

The newtype macros is not in scope.  I don't know enough rust to debug.  Since all the other tests pass, I defaulted to skip tests.

Comment 2 Stuart D Gathman 2022-10-13 01:10:41 UTC
Added --skip to %cargo_test

%cargo_test -- -- --skip newtype_macros::new_type


test result: ok. 21 passed; 0 failed; 0 ignored; 0 measured; 3 filtered out; finished in 2.47s

error: test failed, to rerun pass '--lib'

Caused by:
  process didn't exit successfully: `/builddir/build/BUILD/sodiumoxide-0.2.7/target/release/deps/sodiumoxide-730831ba5980a0f8 --skip 'newtype_macros::new_type'` (signal: 4, SIGILL: illegal instruction)


Seems like a bug in %cargo_test macro?

Comment 3 Fabio Valentini 2022-10-13 09:42:38 UTC
> Seems like a bug in %cargo_test macro?

Why would a crashing test be a bug in the %cargo_test macro?

The test runner is killed with "SIGILL" if the code of the test itself can't be executed. Recent versions of Rust / LLVM will emit "ud2" instructions on x86_64 if they encounter undefined behaviour. So I'd look at the code for the test that crashes the test runner to see how UB can happen.

Comment 4 Stuart D Gathman 2022-10-13 16:54:24 UTC
It already printed the summary.  All tests passed and 3 were skipped.  How would you know which test got a UD?

Comment 5 Fabio Valentini 2022-10-13 17:44:17 UTC
> It already printed the summary.  All tests passed and 3 were skipped.  How would you know which test got a UD?

The problem is that if a test crashes instead of reporting failure, it will not be listed, since the test runner can't report back.
You can try running tests with "--test-threads 1" to make things run sequentially, and then it should be obvious which test's thread crashed the test runner.

I tried reproducing the failure locally from a source checkout, and the only test (other than the "new_type" doctests failures) is:
crypto::sign::ed25519::test::test_sign_verify_detached_tamper

This might or might not be the failure that causes the SIGILL in your case.

I also noticed that the upstream project was officially abandoned last month :(
https://github.com/sodiumoxide/sodiumoxide

Comment 6 Fabio Valentini 2022-10-15 19:17:01 UTC
I have debugged the test failure I mentioned above. It appears that the test code calls a deprecated API in the ed25519 crate ("Signature::new"), which no longer accepts arbitrary bytes as input, and panics when given invalid data - which is what this test does.

So you can safely skip that test by using something like

# * skip a test that is not compatible with ed25519 1.3.0+:
#   the Signature::new method now panics when called with invalid bytes
# * skip doctests with missing imports for the new_type! macro
%cargo_test -- -- --skip crypto::sign::ed25519::test::test_sign_verify_detached_tamper --skip src/newtype_macros.rs

Please check if you're still seeing the SIGILL crashes when doing that, since I can't reproduce them on my local system (the package builds successfully for me with the two skipped tests).

Comment 7 Stuart D Gathman 2022-12-04 00:26:26 UTC
test result: ok. 21 passed; 0 failed; 0 ignored; 0 measured; 3 filtered out; finished in 2.37s

error: 1 target failed:
    `--lib`

I wonder what that means...

Comment 8 Fabio Valentini 2022-12-04 09:00:28 UTC
Hard to say without more context (i.e. the whole output of %cargo_test).
I assume the actual error is further up, soccer tests are run in this order: Unit tests (--lib), doctests (--doc), and integration tests (--tests). So I assume the test result summary was for doctests, but unit tests further up failed.

Comment 9 Fabio Valentini 2023-03-22 22:48:48 UTC
Please follow up here if you still need this crate (looks like cjdns still needs it?).
Note that the upstream project has been marked as deprecated and unmaintained, though, which probably isn't a good sign for a crypto library.

Comment 10 Stuart D Gathman 2023-03-25 02:39:07 UTC
Cjdns new version need the crate.  I'm not sure what Rust programmers use if not a libsodium wrapper.  I'll ask cjdns upstream about the sodiumoxide status.

Comment 11 Stuart D Gathman 2023-03-25 02:39:59 UTC
I have to resubmit a bunch of the dependencies to get this review finished.

Comment 12 Fabio Valentini 2023-05-02 22:13:59 UTC
I can help with submitting missing dependencies, if that would help you ...

Comment 13 Stuart D Gathman 2023-05-04 20:31:19 UTC
Yes, please.  I just got cjdns-21.1 finally fixed (using distro flags).  Caleb upstream is aware of the conflict between wrapping libsodium and rewriting crypto stuff in Rust.  The Rust equivalents aren't as fast or complete (yet?).  A wrapper only tracks the API, so is more stable.

Comment 14 Fabio Valentini 2023-05-05 14:30:54 UTC
Ok, that would be a package for the "ed25519" crate, and its dependencies?
I will work on that as time permits.

Comment 15 Fabio Valentini 2023-05-19 15:56:14 UTC
The ed25519 crate is now available from Fedora. Can you refresh this package so I can continue with the review?

Comment 16 Stuart D Gathman 2023-05-26 22:22:06 UTC
It is failing a new test:
test crypto::aead::aes256gcm::aes_api::test::test_seal_open_detached ... FAILED

That is from rust-libsodium_sys

Looking into it

Comment 17 Stuart D Gathman 2023-05-26 22:39:00 UTC
Skipping the test gets this:

  process didn't exit successfully: /builddir/build/BUILD/sodiumoxide-0.2.7/target/release/deps/sodiumoxide-be316f85c0acbe9e --skip 'crypto::sign::ed25519::test::test_sign_verify_detached_tamper' --skip src/newtype_macros.rs --skip 'crypto::aead::aes256gcm::aes_api::test::test_seal_open_detached' (signal: 4, SIGILL: illegal instruction)

Comment 18 Fabio Valentini 2023-05-27 11:16:12 UTC
This means that either the code is invalid and gets simplified down to an "ud2" x86 instruction by LLVM (i.e. "this is invalid"), or you're running code that requires a newer instruction set (like AVX2 or AVX512) on an older CPU that doesn't support it.

Comment 19 Fabio Valentini 2023-06-08 19:37:16 UTC
I tried a local build with

%cargo_test -- -- --skip crypto::sign::ed25519::test::test_sign_verify_detached_tamper --skip src/newtype_macros.rs

And the build succeeded without errors. Is it possible that the CPU feature detection in libsodium is broken and / or the CPU in your development machine is old?

Comment 20 Fabio Valentini 2023-07-30 16:30:21 UTC
Any progress?
If you prefer, we could also close this review so somebody else can continue with a fresh one.

Comment 21 Stuart D Gathman 2023-07-31 14:28:21 UTC
Still getting the same illegal instruction error as comment 2.  I'll try your suggested skip from comment 19, and then a scratch build.  

Is this an "old CPU"?  All my computers are discards.  :-)

Vendor ID:               GenuineIntel
  Model name:            Intel(R) Core(TM) i3-2120 CPU @ 3.30GHz

I have some other machines to try a local build on.

Comment 22 Stuart D Gathman 2023-07-31 14:37:59 UTC
Trying local build with   Model name:            Intel(R) Core(TM) i5-2520M CPU @ 2.50GHz

Comment 23 Fabio Valentini 2023-07-31 14:48:45 UTC
Yeah those are "pretty old" :)
Just a guess, the tests might exercise some code paths of libsodium that use AVX2 extensions.
AVX2 has only been implemented in the 4000-series of Intel CPUs and newer, which might explain the problem (but just an educated guess).

Comment 24 Stuart D Gathman 2023-07-31 15:21:28 UTC
Spec URL: https://gathman.org/linux/SPECS/rust-sodiumoxide.spec
SRPM URL: https://gathman.org/linux/SRPMS/rust-sodiumoxide-0.2.7-4.fc38.src.rpm

Builds normally on i5 cpu.  I'd call that a bug in libsodium most likely.

Comment 25 Fedora Review Service 2023-07-31 15:29:53 UTC
Copr build:
https://copr.fedorainfracloud.org/coprs/build/6225986
(succeeded)

Review template:
https://download.copr.fedorainfracloud.org/results/@fedora-review/fedora-review-2121490-rust-sodiumoxide/fedora-rawhide-x86_64/06225986-rust-sodiumoxide/fedora-review/review.txt

Please take a look if any issues were found.

---
This comment was created by the fedora-review-service
https://github.com/FrostyX/fedora-review-service

If you want to trigger a new Copr build, add a comment containing new
Spec and SRPM URLs or [fedora-review-service-build] string.

Comment 26 Fabio Valentini 2023-07-31 16:17:29 UTC
Thanks for the update, I submitted a scratch build:
https://koji.fedoraproject.org/koji/taskinfo?taskID=104183424

The package seems to build as expected on x86_64 and i686, but it fails on non-x86 architectures
(aarch64, ppc64le, s390x) because the code uses x86-specific Rust macros:

error: This macro cannot be used on the current target.
                                   You can prevent it from being used in other architectures by
                                   guarding it behind a cfg(any(target_arch = "x86", target_arch = "x86_64")).
   --> src/crypto/aead/aes256gcm.rs:198:17
    |
198 |                 is_x86_feature_detected!("aes") && is_x86_feature_detected!("pclmulqdq");
    |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: this error originates in the macro `is_x86_feature_detected` (in Nightly builds, run with -Z macro-backtrace for more info)
error: This macro cannot be used on the current target.
                                   You can prevent it from being used in other architectures by
                                   guarding it behind a cfg(any(target_arch = "x86", target_arch = "x86_64")).
   --> src/crypto/aead/aes256gcm.rs:198:52
    |
198 |                 is_x86_feature_detected!("aes") && is_x86_feature_detected!("pclmulqdq");
    |                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: this error originates in the macro `is_x86_feature_detected` (in Nightly builds, run with -Z macro-backtrace for more info)

Since this is only affecting code for a test, it should be simple to fix.

My suggestion would be to patch line 193 of src/crypto/aead/aes256gcm.rs
from
#[cfg(feature = "std")]
to
#[cfg(all(feature = "std", any(target_arch = "x86", target_arch = "x86_64")))]

Attaching a patch with this change to the spec file fixes compilation issues on aarch64, ppc64le and s390x, but reveals new test failures on these architectures:
https://koji.fedoraproject.org/koji/taskinfo?taskID=104184002

---- crypto::aead::aes256gcm::aes_api::test::test_seal_open stdout ----
thread 'crypto::aead::aes256gcm::aes_api::test::test_seal_open' panicked at 'called `Result::unwrap()` on an `Err` value: ()', src/crypto/aead/aes256gcm.rs:206:40
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
---- crypto::aead::aes256gcm::aes_api::test::test_seal_open_detached stdout ----
thread 'crypto::aead::aes256gcm::aes_api::test::test_seal_open_detached' panicked at 'called `Result::unwrap()` on an `Err` value: ()', src/crypto/aead/aes256gcm.rs:222:40
---- crypto::aead::aes256gcm::aes_impl::test::test_vector_1 stdout ----
thread 'crypto::aead::aes256gcm::aes_impl::test::test_vector_1' panicked at 'assertion failed: `(left == right)`
  left: `0`,
 right: `54`', src/crypto/aead/aes256gcm.rs:74:13
failures:
    crypto::aead::aes256gcm::aes_api::test::test_seal_open
    crypto::aead::aes256gcm::aes_api::test::test_seal_open_detached
    crypto::aead::aes256gcm::aes_impl::test::test_vector_1
test result: FAILED. 248 passed; 3 failed; 0 ignored; 0 measured; 1 filtered out; finished in 17.30s

Not sure how you want to deal with these.
They don't look "harmless" so it might or might not be a good idea to just skip these tests.

Comment 27 Fabio Valentini 2023-12-04 15:35:54 UTC
Can you follow up here please?

- Do you still need this package?
- Is cjdns considering to move to a different crypto library?

Comment 28 Stuart D Gathman 2023-12-09 23:12:57 UTC
I've mentioned the unsupported status of sodiumoxide to CJD, but he is working on PKT coin (Cjdns integrated with proof of bandwidth for a blockchain currency to reward local network links for anyone).

I do need to keep working on getting sodiumoxide (and new versions of cjdns) built.  Sorry I've been so busy with other stuff.


Note You need to log in before you can comment on or make changes to this bug.