Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 915043 - Package rust (lang)
Summary: Package rust (lang)
Keywords:
Status: CLOSED DUPLICATE of bug 1356907
Alias: None
Product: Fedora
Classification: Fedora
Component: distribution
Version: rawhide
Hardware: Unspecified
OS: Unspecified
high
unspecified
Target Milestone: ---
Assignee: Václav Pavlín
QA Contact: Radek Vokál
URL:
Whiteboard:
Depends On: 1111834 hoedown
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-02-24 11:17 UTC by Fabian Deutsch
Modified: 2018-08-22 08:08 UTC (History)
76 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-07-28 10:33:05 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Fabian Deutsch 2013-02-24 11:17:42 UTC
Description of problem:
Mozilla's rust lang (http://rust-lang.org) is not yet packaged for Fedora. As the language itself is quite inetersting I'm adding this bug to track packaging efforts.

LLnextgen is an optional tool (which also isn't packaged) but should be fairly easy to package.

Comment 1 Fabian Deutsch 2013-03-08 10:54:13 UTC
Upstream bugs:
How to handle rpath: https://github.com/mozilla/rust/issues/5219
LLVM patches: https://github.com/mozilla/rust/issues/4259

Comment 2 Fabian Deutsch 2013-03-08 10:54:42 UTC
Syntax highlighting for gedit:
https://github.com/mozilla/rust/tree/master/src/etc/gedit

Comment 3 Fabian Deutsch 2013-03-08 10:56:51 UTC
A draft spec:
https://github.com/fabiand/rust-spec

Can't be included in Fedora yet because of non-conformance with the guidelines.

Comment 4 Fabian Deutsch 2013-03-11 08:59:31 UTC
Maybe rust should be packaged as a software collection [0]

[0] https://fedoraproject.org/wiki/SoftwareCollections

Comment 5 Fabian Deutsch 2013-03-26 19:23:51 UTC
Additionally I noted that 
1. libuv [1] is required and needs to be packaged too.
2. A bootstrap binary (rustc) is required

[1] https://github.com/joyent/libuv

Comment 6 Fedora End Of Life 2013-04-03 16:34:07 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 19 development cycle.
Changing version to '19'.

(As we did not run this process for some time, it could affect also pre-Fedora 19 development
cycle bugs. We are very sorry. It will help us with cleanup during Fedora 19 End Of Life. Thank you.)

More information and reason for this action is here:
https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora19

Comment 7 Fabian Deutsch 2013-05-23 13:56:23 UTC
(In reply to Fabian Deutsch from comment #5)
> Additionally I noted that 
> 1. libuv [1] is required and needs to be packaged too.

There is a release of libuv now:
https://github.com/joyent/libuv/issues/354#issuecomment-17803641

Comment 8 Rick Elrod 2013-09-25 06:06:42 UTC
Hi, is there any status update on this?

libuv is in Fedora: https://apps.fedoraproject.org/packages/libuv

I guess we are still waiting on upstream to get their patches back in LLVM? Or is there something else this is blocked by?

Comment 9 Fabian Deutsch 2013-09-25 06:22:37 UTC
Hey Ricky,

yes - the llvm patches are still not upstreamed, which is in turn blocking the packaging for Fedora.
There seems to be some movement around the upstreaming, take a look here: https://github.com/mozilla/rust/issues/4259

But we can already start to unbundle libuv in the existing rust pkg which is kept here: https://github.com/fabiand/rust-spec

Comment 10 Fabian Deutsch 2013-09-27 13:21:04 UTC
I've just updated the spec to build rust-0.8.
A build is also triggered here: http://copr-fe.cloud.fedoraproject.org/coprs/detail/fabiand/rust/builds/

A local build went fine.

Comment 13 Andy Grover 2014-07-30 21:27:10 UTC
Looks like llvm changes that Rust needs are in LLVM 3.5. Setting this to depend on bug 1123103.

Comment 14 Christopher Meng 2014-08-02 05:32:30 UTC
I've added some changes to the fab's spec, also interested on rust.

Comment 15 Andy Grover 2014-11-06 20:08:11 UTC
rawhide has llvm-3.5.0-2 now.

Comment 16 Fabian Deutsch 2014-11-11 09:49:36 UTC
LLVM might finally work, but looking at the submodules list:

> [submodule "src/llvm"]
> path = src/llvm
> url = https://github.com/rust-lang/llvm.git
>  branch = master

Might work these days

> [submodule "src/compiler-rt"]
> path = src/compiler-rt
> url = https://github.com/rust-lang/compiler-rt.git

Not packaged yet

> [submodule "src/rt/hoedown"]
> path = src/rt/hoedown
> url = https://github.com/rust-lang/hoedown.git

Not packaged yet

> [submodule "src/jemalloc"]
> path = src/jemalloc
> url = https://github.com/rust-lang/jemalloc.git

Needs a static subpackage afaiu



I wonder if we should make this a change for F22 or 23.

Then we have a goal, and could track packaging the dependencies, as well as adding the static subpackages.

Comment 17 Christopher Meng 2014-11-15 05:01:40 UTC
(In reply to Fabian Deutsch from comment #16)
> LLVM might finally work, but looking at the submodules list:
> 
> > [submodule "src/llvm"]
> > path = src/llvm
> > url = https://github.com/rust-lang/llvm.git
> >  branch = master
> 
> Might work these days

Or might not, as the upstream told me some changes are not merged into LLVM upstream.

Comment 18 Daniel Miranda 2015-04-25 23:12:27 UTC
I suppose this hasn't got into F22. But now that Rust is nearing 1.0 it would be very nice to have it in F23. Is the situation with LLVM still unresolved? The issue in the Rust bugtracker seems to indicate only one optional patch was still missing, is that still the case?

Comment 19 Fabian Deutsch 2015-04-27 09:34:34 UTC
According to my knowledge all important patches are in upstream LLVM.

So, the current state is still as it was in comment 16.

The easiest way to start is to find volunteers to package parts of the dependencies.

Comment 20 Seo Sanghyeon 2015-04-28 12:43:51 UTC
Upstream here. Rust builds and passes all tests with LLVM 3.6 release, and Debian package in progress is using system LLVM fine. Here is a short howto:

https://internals.rust-lang.org/t/building-with-system-llvm/1647

Comment 21 Fabian Deutsch 2015-04-28 21:08:45 UTC
Nice to see comment 20.

Just a heads up, I packaged hoedown and compiler-rt. I'll push the specs tomorrow, including opening the review requests.

Comment 22 Josh Stone 2015-04-28 21:32:57 UTC
Great!  I'd be happy to act as reviewer, and even co-maintainer if you like...

Comment 23 Fabian Deutsch 2015-04-29 17:13:59 UTC
I've added the two specs to this repository:

https://github.com/fabiand/rust-spec

hoedown and compiler-rt build for me locally.
I'm now in progress in updating the rust spec to work with beta3.

Comment 24 Josh Stone 2015-04-29 18:08:50 UTC
Your hoedown.spec builds fine for me, but compiler-rt gives me errors.
With mock -r fedora-rawhide-x86_64, I get:

/builddir/build/BUILD/compiler-rt-a33d046b2491adfe05655bc252037ab260f9c485/lib/builtins/atomic.c:35:53: error: extra tokens at end of '#pragma redefine_extname' - ignored [-Werror,-Wignored-pragmas]
#pragma redefine_extname __atomic_load_c SYMBOL_NAME(__atomic_load)
                                                    ^
/builddir/build/BUILD/compiler-rt-a33d046b2491adfe05655bc252037ab260f9c485/lib/builtins/atomic.c:36:54: error: extra tokens at end of '#pragma redefine_extname' - ignored [-Werror,-Wignored-pragmas]
#pragma redefine_extname __atomic_store_c SYMBOL_NAME(__atomic_store)
                                                     ^
/builddir/build/BUILD/compiler-rt-a33d046b2491adfe05655bc252037ab260f9c485/lib/builtins/atomic.c:37:57: error: extra tokens at end of '#pragma redefine_extname' - ignored [-Werror,-Wignored-pragmas]
#pragma redefine_extname __atomic_exchange_c SYMBOL_NAME(__atomic_exchange)
                                                        ^
/builddir/build/BUILD/compiler-rt-a33d046b2491adfe05655bc252037ab260f9c485/lib/builtins/atomic.c:38:65: error: extra tokens at end of '#pragma redefine_extname' - ignored [-Werror,-Wignored-pragmas]
#pragma redefine_extname __atomic_compare_exchange_c SYMBOL_NAME(__atomic_compare_exchange)
                                                                ^
4 errors generated.

Comment 25 Josh Stone 2015-05-02 00:40:46 UTC
It seems the clang preprocessor is not expanding SYMBOL_NAME in that pragma line.  I can run "clang -E" on it and see the pragma is untouched, but "gcc -E" expands it.  If I manually expand SYMBOL_NAME, which on Linux is a no-op, then clang compiles the rest fine.

Anyway, I think we may not even compiler-rt as a separate package.  If you look at llvm.spec, you'll see it already has compiler-rt sources.  It's shipped in clang.rpm as "/usr/lib/clang/3.6.0/lib/linux/libclang_rt.builtins-x86_64.a".  I confirmed this file has the exact same symbols as a manually compiled libcompiler_rt.a.

So perhaps we can just patch rust/mk/rt.mk to pull clang's copy?

Comment 26 Josh Stone 2015-05-02 00:58:36 UTC
On hoedown, why does it BuildRequire clang?  It appears to use plain $(CC) to build.  Also, I think it would be better to patch the Makefile to input _bindir, _includedir, and _libdir, rather than manually moving /lib/* to _libdir.

Comment 27 Colin Walters 2015-05-03 15:38:27 UTC
Unrelated to actual packaging, in case anyone finds it useful, I've updated Fabian's spec file for rust-binary here for the latest release:

https://github.com/cgwalters/playground/blob/master/rust/rust-binary.spec

Comment 28 Fabian Deutsch 2015-05-04 10:56:49 UTC
(In reply to Josh Stone from comment #25)
> It seems the clang preprocessor is not expanding SYMBOL_NAME in that pragma
> line.  I can run "clang -E" on it and see the pragma is untouched, but "gcc
> -E" expands it.  If I manually expand SYMBOL_NAME, which on Linux is a
> no-op, then clang compiles the rest fine.
> 
> Anyway, I think we may not even compiler-rt as a separate package.  If you
> look at llvm.spec, you'll see it already has compiler-rt sources.  It's
> shipped in clang.rpm as
> "/usr/lib/clang/3.6.0/lib/linux/libclang_rt.builtins-x86_64.a".  I confirmed
> this file has the exact same symbols as a manually compiled libcompiler_rt.a.
> 
> So perhaps we can just patch rust/mk/rt.mk to pull clang's copy?

Josh, I did not notice that llvm is bundling those sources, but yes, if it does then I think we can reuse and point rust to it.

Comment 29 Zeeshan Ali 2015-05-23 15:53:32 UTC
Fabian, Any updates on this? Now that rust is 1.0, would be really nice to be able to get rust installed easily cause I'm sure many people have been waiting for it to become stable before they look into it.

Comment 30 Fabian Deutsch 2015-05-23 21:05:08 UTC
Zeeshan, no - no update on this. Contributions are welcome.

It has just some many loose ends that it's hard to get by.

Comment 31 David Voit 2015-05-24 00:06:34 UTC
Hi All,

i tried to get this running, but i'm still not there.

https://github.com/davidvoit/rust/tree/fedora - has my current rust side work on this.

Maybe this helps. Currently not all stages get's through here...

The Makefiles also need patching for miniz and jemalloc still. What about libbacktrace?

Greetings
David

Comment 32 Josh Stone 2015-05-26 16:31:52 UTC
FWIW, I submitted a PR on hoedown to improve its packaging:
https://github.com/hoedown/hoedown/pull/160

Comment 33 Christian Stadelmann 2015-05-28 11:27:24 UTC
Rust has no ABI and no library versioning yet [1]. Isn't this blocking packaging rust for fedora? You would have to rebuild every rust-related package with every rust update. As much as I would like seeing rust in Fedora I don't think rust is ready yet because of these limitations.

[1] https://github.com/rust-lang/rfcs/issues/600

Comment 34 Josh Stone 2015-05-28 16:14:44 UTC
You're right that libraries have no ABI for Rust code -- we might even want to prune rustlib/$target/lib/*.so from the compiler package.  Yet I still think it's worth having rustc and cargo in Fedora as a first step.

FWIW, you could ship a rust library sharing only "#[no_mangle] pub extern" functions.  They will have C calling conventions, for an FFI from other languages.

Comment 35 Josh Stone 2015-07-07 00:49:01 UTC
They are considering a change to the default allocator:
https://github.com/rust-lang/rfcs/pull/1183

I think for Fedora we'd prefer the system malloc (from glibc) most of the time.  See also the note at the bottom about maybe using jemalloc without a prefix, which is how Fedora's jemalloc is built already.

So this change may simplify one dependency for us.

Comment 36 Colin Walters 2015-08-11 22:06:21 UTC
While this is being worked on, I've updated:

https://copr.fedoraproject.org/coprs/fabiand/rust-binary/

Comment 37 Mattias Bengtsson 2015-09-11 04:28:49 UTC
FYI: https://internals.rust-lang.org/t/perfecting-rust-packaging/2623

Comment 38 Igor Gnatenko 2015-10-16 12:08:21 UTC
Hi folks, I think we need to go forward with this ;)

Comment 39 Josh Stone 2015-10-16 17:07:35 UTC
From the conversation linked in comment #37 came The Plan:
https://internals.rust-lang.org/t/perfecting-rust-packaging-the-plan/2767

I think we at least need the bootstrapping TODOs done, so we can get to an srpm without any snapshot binary.

Comment 40 Balakrishnan Balasubramanian 2016-01-30 07:51:59 UTC
+1 rust has reached version 1.6!

Comment 41 Guillaume Bonnet 2016-02-03 16:19:06 UTC
I made a package for my personal use. It is mostly based on Fabian's one, but is updated to work with Rust 1.6. I made it available at:
https://github.com/gmbonnet/fedora-rust-spec

This package does not conform to the Guidelines yet, since LLVM, jemalloc and hoedown are not unbundled.

I made three subpackages: rust, rust-doc and rust-gdb. I also opened some pull requests on GitHub because patches were needed:
https://github.com/rust-lang/rust/pull/31169
https://github.com/rust-lang/rust/pull/31074
https://github.com/rust-lang/rust/pull/31388

There is fourth pull request opened about soname handling:
https://github.com/rust-lang/rust/pull/31170

Since Rust has no binary compatibility, it does not make much sense to add sonames to binaries however. It would be great to have an official policy about inclusion of soname in Rust libraries (the pull request will remain unmerged until we have this policy). Maybe should we contact the Packaging Committee?

Comment 42 Richard Bradfield 2016-02-03 16:26:07 UTC
Is there any point in developing a spec file for future inclusion in Fedora while the build process still requires the Stage0 snapshot?

My interpretation of the Guidelines is the snapshot is permitted for the very first build, but you must immediately bump the spec and build a new package with the compiler from the first binary RPM.

Is this correct or are there other loopholes?

Comment 43 Colin Walters 2016-02-03 16:35:48 UTC
(In reply to Guillaume Bonnet from comment #41)
> I made a package for my personal use. It is mostly based on Fabian's one,
> but is updated to work with Rust 1.6. 

The COPR is now updated to 1.6:
https://copr.fedorainfracloud.org/coprs/fabiand/rust-binary/
https://copr.fedorainfracloud.org/coprs/fabiand/rust-binary/build/157722/

> I made three subpackages: rust, rust-doc and rust-gdb. I also opened some
> pull requests on GitHub because patches were needed:
> https://github.com/rust-lang/rust/pull/31169
> https://github.com/rust-lang/rust/pull/31074
> https://github.com/rust-lang/rust/pull/31388

Let's collaborate on updating the spec file?  I'm not sure why the COPR doesn't have dist-git...maybe since it was created before that?

Anyways we can use the list of SRPMs as an extremely lame version control system, or alternatively start doing PRs against
https://github.com/fabiand/rust-spec

?

Comment 44 Guillaume Bonnet 2016-02-03 16:45:11 UTC
To Richard:

It's true that Rust is not yet ready for inclusion in Fedora, but I think that it makes sense to resolve other packaging issues when they arise (although it is not urgent).

There is a possible workaround for the bootstrapping problem (this is just an idea I had, I did not test it yet). We could build ourselves the latest stage0 archive and package it in a `rust-bootstrap` package. It would then be possible to update `rust-bootstrap` whenever a new stage0 compiler is released, and use the right version of `rust-bootstrap` to bootstrap the stable `rust` package.

Comment 45 Richard Bradfield 2016-02-03 16:49:13 UTC
@Guillaume,

Sounds good, the unbundling of jemalloc and so on is a worthwhile effort at any rate.

From a discussion on the Rust IRC, it should be possible to package unstable Rust, unstable versions are fully capable of building their successor, however they use features that are gated in the Stable release channel (thus preventing Stable from building itself).

Is there a place in Rawhide for packages that are absolutely not ready to go out in a release, but are still usable for people who know what they are doing?

Comment 46 Guillaume Bonnet 2016-02-03 16:52:25 UTC
@Colin:

At the time I wrote my spec file, I was just aware of the outdated version at https://github.com/fabiand/rust-spec, but it would be great, of course, to merge my version in more well-established spec files. I'll download the version from COPR to not duplicate packaging effort.

Comment 47 Guillaume Bonnet 2016-02-03 17:11:23 UTC
@Richard,

Are you looking for a place for packages like `rust-bootstrap`? The problem is that if we want to distribute `rust` in a stable release, Koji will need to have `rust-bootstrap` in that release too, since it would be a dependency of `rust`.

I think that the Go packaging team (https://fedoraproject.org/wiki/PackagingDrafts/Go) has been facing a similar problem, since they package source code of Go libraries in stable releases, although end-users are not supposed to install those packages (they are just build-time dependencies).

In addition, `rust-bootstrap` would not conflict with `rust`, since it would not contain a full installation of Rust but just stage0 archives similar to the ones distributed by Rust developers. This is as close to upstream as possible, and just permits as to bootstrap Rust itself (it would not be possible, for example, to compile some random program written in Rust using only `rust-bootstrap`).

Comment 48 Link Dupont 2016-02-03 17:28:27 UTC
For IDEs that want to use rust-racer for code completion, we'll need a package that contains the rust source (call it rust-src). Would it be possible to build a stage0 from those sources during the %build of rustc and family? That way the spec can BuildRequires: rust-src, a package that has value outside of simply bootstrapping rustc.

Comment 49 Guillaume Bonnet 2016-02-03 19:47:01 UTC
Thank you for your feedback @Link.

I don't think that packaging the Rust source code will be enough for our problem (to build a stage0, we need the previous version of this stage0, so we still need a binary).

However, packaging rust-src is still something we should care about if rust-racer needs it. How does rust-racer use Rust's source code exactly? Does it need a particular version of this source code? Maybe rust-src should be a subpackage of rust. Are sources of Rust crates also needed by rust-racer?

If we eventually do not use rust-src for bootstrapping, I guess we will have some extra time to handle this, since rust-racer is probably not yet used by packages that are in official repositories, and we have no policy for packaging Rust crates yet.

Comment 50 Link Dupont 2016-02-03 21:01:59 UTC
(In reply to Guillaume Bonnet from comment #49)
> However, packaging rust-src is still something we should care about if
> rust-racer needs it. How does rust-racer use Rust's source code exactly?
> Does it need a particular version of this source code? Maybe rust-src should
> be a subpackage of rust. Are sources of Rust crates also needed by
> rust-racer?
> 
> If we eventually do not use rust-src for bootstrapping, I guess we will have
> some extra time to handle this, since rust-racer is probably not yet used by
> packages that are in official repositories, and we have no policy for
> packaging Rust crates yet.

I've only just started exploring racer, not actually used or packaged it. But from its github page, it looks like it can be installed as a crate. It appears to require setting an env variable RUST_SRC_PATH that points to a path on disk where the rust sources are kept. I'd guess that, depending on how the crate policy ends up, we could just leave it as an exercise for the user to download the sources and set their own RUST_SRC_PATH if they want to install racer.

https://github.com/phildawes/racer

Comment 51 Josh Stone 2016-02-15 01:05:42 UTC
(In reply to Josh Stone from comment #35)
> See also the note at the bottom about maybe using jemalloc without a
> prefix, which is how Fedora's jemalloc is built already.

FYI, rust's use of jemalloc was just un-prefixed:
https://github.com/rust-lang/rust/pull/31460

Comment 52 Fabian Deutsch 2016-03-23 20:50:26 UTC
The CC list of this six-digit bug is growing.

Are there any valuable recommendations how we can package rust in Fedora?

How was it done for Go? Does Go also bundle so many libraries?

If we accept the bundling and express the bundles in the spec, who has time to work on this?

Comment 53 Richard Bradfield 2016-03-24 10:26:35 UTC
For sake of argument, would anyone oppose the bundling of the unpackaged dependencies, along with a self-hosting build of the Nightly compiler?

I'm still of the opinion we should wait until the stable compiler can build itself before we include it in a Fedora release. On the other hand, something clearly marked 'rust-nightly' could be updated frequently in Rawhide, but we would need to be prepared for backwards incompatible changes.

I have time to work on this, if a consensus can be reached on how to proceed.

Comment 54 Guillaume Bonnet 2016-03-24 12:00:25 UTC
I think a possible plan to handle Rust packaging would be to base our work on the rust-binary package in COPR and to follow the following steps:

1) Choose an efficient way to submit patches/pull requests.

2) Agree on what to put in sections like %install and %post, as well as on which subpackages to create.

3) Adapt the package to build Rust from source, with a prebuild stage0 compiler listed as a source for this RPM.

4) Work on unbundling dependencies (at least those that are already in Fedora, like LLVM).

5) Adapt our work to create a `rust-bootstrap' package.

6) In the main `rust' package, add a dependency to `rust-bootstrap' and remove the prebuild stage0 compiler from the list of sources.

Comment 55 Cesar Eduardo Barros 2016-03-24 12:48:57 UTC
(In reply to Richard Bradfield from comment #53)
> I'm still of the opinion we should wait until the stable compiler can build
> itself before we include it in a Fedora release.

My worry is about Firefox. From what I understand, it is their intention to gradually replace some of its pieces with code written in Rust. When it becomes required for building Firefox, it would be better if a working stable Rust package were ready, instead of having to do it in a rush.

It is also my understanding that Firefox already can use a few bits of code written in Rust, as an optional replacement for some older C++ code. It would be good to be able to use these replacements, since they are supposed to be safer (and to reduce divergence with upstream, if the binaries they distribute are configured to use these replacements).

Relevant Mozilla bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1135640

Comment 56 Christian Stadelmann 2016-03-24 13:44:43 UTC
(In reply to Guillaume Bonnet from comment #54)
> 4) Work on unbundling dependencies (at least those that are already in
> Fedora, like LLVM).

Already done according to this upstream bug report: https://github.com/rust-lang/rust/issues/13945

Still, libbacktrace (C, not packaged in Fedora) and several rust libraries (also not packaged in Fedora) are bundled.

Comment 57 Josh Stone 2016-03-28 16:37:17 UTC
(In reply to Fabian Deutsch from comment #52)
> How was it done for Go? Does Go also bundle so many libraries?

AIUI Go in Fedora is only packaging libraries as source/devel packages to be used in BuildRequires, and then final executables are statically linked.  We discussed this as likely the best plan in "Perfecting Rust Packaging" too.
https://fedoraproject.org/wiki/PackagingDrafts/Go

(In reply to Richard Bradfield from comment #53)
> I'm still of the opinion we should wait until the stable compiler can build
> itself before we include it in a Fedora release.

I think it may be possible to kludge past this with CFG_BOOTSTRAP_KEY, especially if we pick a static string for Fedora use, as I noted here:
https://github.com/rust-lang/rust/issues/29557#issuecomment-167680241

(In reply to Christian Stadelmann from comment #56)
> several rust libraries (also not packaged in Fedora) are bundled.

I know only of liblibc -- are there others?  IMO this one is special enough and so integral to the standard library that it deserves an exception.  Even when liblibc is packaged in Fedora, that will be some release version, where as libstd is expecting specific snapshots.  Anyway, liblibc is only some type definitions and function declarations, so it's not very scary to bundle.

Comment 58 Andy Grover 2016-04-26 20:47:53 UTC
FWIW according to:

https://this-week-in-rust.org/blog/2016/04/25/these-weeks-in-rust-127/

Rust bootstraps from previous stable release now:

https://github.com/rust-lang/rust/pull/32942

however I think this just got merged on the unstable branch :) so we need to wait 12 weeks for this to actually be *in* the stable Rust release, where we could take advantage of it.

Comment 59 Josh Stone 2016-04-26 21:03:05 UTC
Yes, and I'm working on bootstrapping from the current release:
https://github.com/rust-lang/rust/issues/29556

Comment 60 Josh Stone 2016-04-26 22:52:27 UTC
If someone wants to work on hoedown packaging, I think that should be a relatively easy interim task.  Then Rust still needs to learn how to use an external build.

The other bundled git-submodules are:

- llvm: already possible to configure --llvm-root=/usr

- compiler-rt: Fedora used to be package this with clang, now it has its own package.  Rust still needs work to enable using an external build.

- jemalloc: there is configure --jemalloc-root=/path, but this only knows how to use static libjemalloc_pic.a.  I think we should use --disable-jemalloc.

- liblibc: Just bindings - I think this is acceptable to bundle.

- rust-installer: I think this is only used at build/install time, not packaged, so it should be acceptable.


Direct bundles that aren't a git-submodule:

- src/libbacktrace/: not packaged in Fedora, and has a few Rust-specific fixes.  FWIW I found that Fedora's "apitrace" package also provides bundled(libbacktrace).

- src/rt/miniz.c: this is packaged as miniz, so perhaps Rust should use it.

- src/librustdoc/html/static/jquery-2.1.4.min.js: rustdoc embeds this directly in itself with "include_bytes!", then writes it out as jquery.js whenever it generates documentation.  Fedora's js-jquery is 2.1.3, but even if that's enough it will take some patching to get rustdoc to use it externally.


That's all the bundling I've found.  It may not be exhaustive.

PS- libuv was mentioned way back in comment #5, but that's been removed since rust 0.12.0 when green threads were dropped.

Comment 61 alsoijw 2016-05-11 17:09:00 UTC
When will be added package?

Comment 62 Christian Stadelmann 2016-05-12 06:52:18 UTC
(In reply to Josh Stone from comment #57)
> (In reply to Christian Stadelmann from comment #56)
> > several rust libraries (also not packaged in Fedora) are bundled.
> 
> I know only of liblibc -- are there others?  IMO this one is special enough
> and so integral to the standard library that it deserves an exception.  Even
> when liblibc is packaged in Fedora, that will be some release version, where
> as libstd is expecting specific snapshots.  Anyway, liblibc is only some
> type definitions and function declarations, so it's not very scary to bundle.

You're probably right. https://github.com/rust-lang/rust/tree/master/src lists some more, but they are llvm stuff only.

Comment 63 Josh Stone 2016-07-03 00:28:01 UTC
Now that the bootstrapping stuff is almost here in 1.10, I've taken a stab at this.  Here's my fork of rust-spec and a copr (build in progress):
  https://github.com/cuviper/rust-spec
  https://copr.fedorainfracloud.org/coprs/jistone/rust/

It built fine for me locally, but my first attempt at a source-only rebuild didn't work -- it got a confusing message about missing libcore.  The local rustc should really only be needed as a compiler itself, not for any of its libraries, so I'm not sure what's going on there.  I'll keep poking.

@gmbonnet, sorry I forgot about your spec when I started on my changes.  I definitely did benefit from some of your upstream PRs though, thanks!  It looks like you were further along in subpackaging, so I'll look more at yours later.

For bundled libraries, I'm using Fedora's LLVM and disabling jemalloc.  Igor started the process for hoedown packaging, and we still need to figure out how to use that and the Fedora compiler-rt in the rustc build.

Help is welcome!  I can accept issues and PRs on my github, or we can meet back at fabiand's, or however people want to work.

Comment 64 Demi Marie Obenour 2016-07-09 07:09:24 UTC
Rust should definitely statically link (to other Rust code, not to non-bundled C libraries).  Rust has absolutely no ABI compatibility – changing any crate require that every crate that depends on it be recompiled, transitively; thus, there is little point in dynamically linking Rust code into other Rust code, as it will need to be recompiled just as often as if it were dynamically linked.

Comment 65 Christian Stadelmann 2016-07-09 07:53:32 UTC
(In reply to Demetrios Obenour from comment #64)
> Rust should definitely statically link (to other Rust code, not to
> non-bundled C libraries).  Rust has absolutely no ABI compatibility –
> changing any crate require that every crate that depends on it be
> recompiled, transitively; thus, there is little point in dynamically linking
> Rust code into other Rust code, as it will need to be recompiled just as
> often as if it were dynamically linked.

How should we link to a standard library then? Or is this a rule with exceptions? It could happen that we need to recompile the rust standard library between two major releases (e.g. due to bugs, especially security-related ones).

Instead of linking statically, we should expect to recompile every rust package every week (probably worst case).

On the long term rust needs ABI stability, but it's not there yet. Most important reason: package and disk size.

Comment 66 Guillaume Bonnet 2016-07-09 12:24:36 UTC
Currently Rust and Go handle libraries in similar ways, so I think that we should use similar packaging guidelines. There is information in

https://fedoraproject.org/wiki/PackagingDrafts/Go#Packaging_Libraries

I don't know how the Go static library is linked, however.

Comment 67 Guillaume Bonnet 2016-07-09 12:28:25 UTC
@cupiver Your spec file seems great. We should add `Requires: gcc` to the main package since the Rust compiler uses gcc for linking (or perhaps should we propose a patch to use ld directly).

Comment 68 Josh Stone 2016-07-11 17:54:37 UTC
(In reply to Christian Stadelmann from comment #65)
> How should we link to a standard library then? Or is this a rule with
> exceptions? It could happen that we need to recompile the rust standard
> library between two major releases (e.g. due to bugs, especially
> security-related ones).

In the spec I've written, all Rust libraries are filtered from requires/provides, and I completely removed all "rustlib/$target/lib/*.so".  Only those *.rlib remain, so you'll only be able to statically link the standard library.

However, rustc and rustdoc are still linked dynamically to their libraries in %{_libdir}.  This should be fine as they are always built together, so the ABI will be kept synchronized.

> Instead of linking statically, we should expect to recompile every rust
> package every week (probably worst case).

If we link dynamically, then every dependent *must* rebuild whenever there's any change, and they all must be submitted to bodhi en masse.  Plus we'd have to add strict %{version}-%{release} requires, so dnf knows to keep them in sync.  Any binaries that people built locally would be at risk for breaking.

If we link statically, it's much more relaxed.  Existing binaries continue working, and dependents only need to rebuild if there's a specific fix that needs to be propagated.  Then they can all update through bodhi independently, since there's no runtime dependency.

I'm a believer that dynamic linking is generally preferable, but Rust isn't ready.

> On the long term rust needs ABI stability, but it's not there yet. Most
> important reason: package and disk size.

I actually think those are less important reasons, but we may have different priorities and constraints.  I'm more interested in deploying bug and security fixes in one place -- an ABI-compatible shared library -- rather than suffering a bunch of rebuilds.

(In reply to Guillaume Bonnet from comment #66)
> Currently Rust and Go handle libraries in similar ways, so I think that we
> should use similar packaging guidelines.

Yes, I think source-based libraries are the way to go here.  I'm not even comfortable packaging static rlib (apart from std), because there is surely some ABI to consider in this format as well.  With source-based libraries, the static rebuilds are only needed for the leaf packages that actually ship a binary.

(In reply to Guillaume Bonnet from comment #67)
> @cupiver Your spec file seems great. We should add `Requires: gcc` to the
> main package since the Rust compiler uses gcc for linking (or perhaps should
> we propose a patch to use ld directly).

Thanks!  I'll try to put it up for a formal review soon.

Good point about the runtime dependency on gcc -- I'll add that.  Switching to "ld" also would be nice, so we could just require binutils.  There's an old open bug:
https://github.com/rust-lang/rust/issues/11937

I think I'll make rust-gdb noarch as you had it, but I'm also thinking that rust-doc is probably better to mark as arch-specific, since it's possible some items may be #[cfg(target...)] dependent.  There are definitely os-related specifics in the standard library, which is fine.  Off-hand, I'm not aware of any arch-specific interfaces in std, but it's possible, so it's better to be safe.


Then we need to start on Cargo, and get some Rust packaging guidelines going.  Maybe think about a F25 Self Contained Change, but I'm not sure how close we are...

Comment 69 Tom Tromey 2016-07-11 18:26:27 UTC
(In reply to Josh Stone from comment #68)

> I think I'll make rust-gdb noarch as you had it,

I think it might be better to remove rust-gdb entirely, or make
it an alias for gdb.

rust-gdb exists to update the safe load path; but in the distro context,
this isn't needed, as the rust-specific files can be put in 
/usr/share/gdb/auto-load, which is automatically assumed to be safe
by gdb.

Comment 70 Josh Stone 2016-07-11 19:38:03 UTC
(In reply to Tom Tromey from comment #69)
> I think it might be better to remove rust-gdb entirely, or make
> it an alias for gdb.

Yeah, I'd keep the command present in some form, since upstream and broader internet context mentions it.

> rust-gdb exists to update the safe load path; but in the distro context,
> this isn't needed, as the rust-specific files can be put in 
> /usr/share/gdb/auto-load, which is automatically assumed to be safe
> by gdb.

It also sets PYTHONPATH, which can perhaps be avoided with relative imports, and it adds a -d source path to find the script in the first place.  If we can make all of that work seamlessly, I'm happy to do so.

Comment 71 Christian Stadelmann 2016-07-12 07:13:33 UTC
(In reply to Josh Stone from comment #68)
> Then we need to start on Cargo, and get some Rust packaging guidelines
> going.  Maybe think about a F25 Self Contained Change, but I'm not sure how
> close we are...

Today is the deadline:
https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedoraproject.org/thread/AE6KPCSEGELZHQBHCFZKYVHS3IE5ZEB6/
https://fedoraproject.org/wiki/Releases/25/Schedule

Comment 72 Josh Stone 2016-07-12 18:44:00 UTC
(In reply to Christian Stadelmann from comment #71)
> Today is the deadline:

Yikes!  I'll try to write something up... worst case we'll just bump it to F26.

Comment 73 Matthew Miller 2016-07-12 20:24:25 UTC
(In reply to Cesar Eduardo Barros from comment #55)
> (In reply to Richard Bradfield from comment #53)
> > I'm still of the opinion we should wait until the stable compiler can build
> > itself before we include it in a Fedora release.
> 
> My worry is about Firefox. From what I understand, it is their intention to
> gradually replace some of its pieces with code written in Rust. When it
> becomes required for building Firefox, it would be better if a working
> stable Rust package were ready, instead of having to do it in a rush.

Looks like this is soon: https://hacks.mozilla.org/2016/07/shipping-rust-in-firefox/ says "TL;DR: Starting with Firefox 48, Mozilla is shipping its first production Rust code, with more to come!"

Comment 74 Josh Stone 2016-07-12 21:23:07 UTC
(In reply to Matthew Miller from comment #73)
> Looks like this is soon:
> https://hacks.mozilla.org/2016/07/shipping-rust-in-firefox/ says "TL;DR:
> Starting with Firefox 48, Mozilla is shipping its first production Rust
> code, with more to come!"

Indeed!  It looks like it's still optional though, requiring a line in  .mozconfig, "ac_add_options --enable-rust".  AFAICT it only requires rustc at this time, not cargo or any additional crates.

Comment 75 Christian Stadelmann 2016-07-12 21:58:01 UTC
According to a good blog on firefox [1], linux builds by default already ship rust mp4 parser since ff45.

[1] https://www.soeren-hentzschel.at/rust/firefox-48-besitzt-in-rust-geschriebenen-medien-parser/ , in German. I couldn't find a primary source in English, sorry.

Comment 76 Dennis Gilmore 2016-07-13 06:09:06 UTC
(In reply to Matthew Miller from comment #73)
> (In reply to Cesar Eduardo Barros from comment #55)
> > (In reply to Richard Bradfield from comment #53)
> > > I'm still of the opinion we should wait until the stable compiler can build
> > > itself before we include it in a Fedora release.
> > 
> > My worry is about Firefox. From what I understand, it is their intention to
> > gradually replace some of its pieces with code written in Rust. When it
> > becomes required for building Firefox, it would be better if a working
> > stable Rust package were ready, instead of having to do it in a rush.
> 
> Looks like this is soon:
> https://hacks.mozilla.org/2016/07/shipping-rust-in-firefox/ says "TL;DR:
> Starting with Firefox 48, Mozilla is shipping its first production Rust
> code, with more to come!"

Will we be able to build and ship it though? we have no mp4 patent grant I am aware of and afaik the only use today is a media player.

Comment 77 Felix Schwarz 2016-07-13 07:33:14 UTC
(In reply to Josh Stone from comment #74)
> It looks like it's still optional though, requiring a line in 
> .mozconfig, "ac_add_options --enable-rust".  AFAICT it only requires rustc
> at this time, not cargo or any additional crates.

Just wanted to add the Mozilla bug for making rust mandatory: https://bugzilla.mozilla.org/show_bug.cgi?id=1284816 (no hints in the bug report when this will happen - so I guess there is no need for any haste to get rust in Fedora).


(In reply to Dennis Gilmore from comment #76)
> Will we be able to build and ship it though? we have no mp4 patent grant I
> am aware of and afaik the only use today is a media player.

My understand is that this only affects the MP4 metadata parser (IANAL but I don't think the "MP4" patents apply to the metadata format) and more importantly that the new rust code is just a rewrite of preexisting C++ code (stagefright lib) which was shipped in Fedora before. So I assume there are no legal problems with the new rust code.

Comment 78 Josh Stone 2016-07-15 23:28:20 UTC
I have a review request here:
https://bugzilla.redhat.com/show_bug.cgi?id=1356907

Comment 79 Peter Lemenkov 2016-07-28 10:33:05 UTC
Ok, closing this as a duplicate of bug 1356907 (already reviewed!).

*** This bug has been marked as a duplicate of bug 1356907 ***


Note You need to log in before you can comment on or make changes to this bug.