Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1427604 - RFE: Two-stage installs for bootstrapping multiple architectures
Summary: RFE: Two-stage installs for bootstrapping multiple architectures
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf
Version: rawhide
Hardware: Unspecified
OS: Unspecified
low
low
Target Milestone: ---
Assignee: rpm-software-management
QA Contact: Fedora Extras Quality Assurance
URL: https://github.com/rpm-software-manag...
Whiteboard:
Depends On: 1190854
Blocks: 1440974
TreeView+ depends on / blocked
 
Reported: 2017-02-28 17:25 UTC by Nathaniel McCallum
Modified: 2017-08-11 20:39 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-08-11 20:39:27 UTC
Type: Bug
Embargoed:

Attachments (Terms of Use)

Description Nathaniel McCallum 2017-02-28 17:25:34 UTC 
On Debian, debootstrap has the --foreign and --second-stage options. This allows you to split the bootstrap process into two stages.

Normally to make a system root filesystem you just run:

# debootstrap ...

However, when building a a filesystem for a foreign architecture (ie aarch64 from x64_64), you instead run:

# debootstrap --foreign ...

This command will download all the packages and unpack them in the specified directory.

Once this is complete, you run:

# chroot $root /debootstrap/debootstrap --second-stage

This second stage will run any of the scripts specified in the packages.

The key point is that in between these two stages, qemu-static binaries can be copied into the root to emulate the new architecture.

dnf should support something like this.
Comment 1 Honza Silhan 2017-03-08 12:09:44 UTC 
first stage:
you can install packages to specific dir by setting `--installroot=/path/`.

second stage:
you can fake architecture which DNF thinks is installed by setting $basearch variable in dnf.conf or from DNF Python API.


Everything can by done in one DNF step.
Comment 2 Jaroslav Mracek 2017-03-08 12:10:33 UTC 
Please can you provide an use-case in more detail to better understand how important is your request?
Comment 3 Neal Gompa 2017-07-01 15:04:33 UTC 
This is identical to what I was describing in devel@ recently[1].

Essentially, the missing piece is support for defining rootfs arch and triggering the usage of qemu-user-static so that scripts and stuff run correctly to build the rootfs sanely for a foreign architecture.

Also, in this mode, it needs to download the very bare minimum needed to execute *itself* (or microdnf maybe) within the rootfs to complete the second stage. The first stage must run no scriptlets, and the second stage must run scriptlets using the target architecture emulator.

Richard W.M. Jones actually gave an excellent summary of the multitude of advantages of having this functionality[2].

[1]: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/AIDUL7O4CJPIZB763N444KTKM63JNE5L/

[2]: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/HQXD477WEIUG4VKMBD2BS2OTQS5NHUEG/
Comment 4 Nathaniel McCallum 2017-07-01 20:51:26 UTC 
(In reply to Jaroslav Mracek from comment #2)
> Please can you provide an use-case in more detail to better understand how
> important is your request?

The most significant use case is creating a non-x86_64 container which runs on x86_64.
Comment 5 Nathaniel McCallum 2017-07-01 20:54:16 UTC 
(In reply to Honza Silhan from comment #1)
> first stage:
> you can install packages to specific dir by setting `--installroot=/path/`.
> 
> second stage:
> you can fake architecture which DNF thinks is installed by setting $basearch
> variable in dnf.conf or from DNF Python API.
> 
> 
> Everything can by done in one DNF step.

Faking the architecture as you have proposed means that the binaries which are downloaded and unpacked will be correct. But running the rpm scriptlets will fail because you aren't on the native architecture. This is precisely why it needs to be divided into two stages. The first stage unpacks the rpms into the install root. In between the first and second stage the user prepares the install root for emulation (usually via statically compiled qemu binaries). The second stage runs all the rpm scriptlets (now emulated).
Comment 6 Igor Gnatenko 2017-07-12 13:17:47 UTC 
This is obviously nice and so on, but patches are welcomed. This is big task and it is definitely not focus for DNF team.
Comment 7 Nathaniel McCallum 2017-07-12 15:48:54 UTC 
It might be possible by downloading qemu-user-static from the host architecture manually and installing without dependencies in the chroot followed by kicking off dnf with --installroot. However, there does not appear to be any way to set basearch (at least from the command line). Using --setopt=basearch=armv7hl throws a warning ("Main config did not have a basearch attr. before setopt") twice and then proceeds to install using the host architecture.
Comment 8 Neal Gompa 2017-07-12 22:39:08 UTC 
So, Nathaniel has posted a PR to resolve this: https://github.com/rpm-software-management/dnf/pull/866
Comment 9 Jaroslav Mracek 2017-08-11 20:39:27 UTC 
The patch was released in dnf-2.6.3-1
Note You need to log in before you can comment on or make changes to this bug.