Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.

Bug 1491806

Summary:	Initial Setup should be able to use relevant Anaconda screens without pulling in unrelated dependencies
Product:	[Fedora] Fedora	Reporter:	Martin Kolman <mkolman>
Component:	anaconda	Assignee:	Martin Kolman <mkolman>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	rawhide	CC:	anaconda-maint-list, dlehman, dustymabe, jkonecny, jonathan, kellin, mkolman, pbrobinson, pjones, sbueno, sgallagh, vanmeeuwen+fedora, vponcova, vtrefny, wwoods
Target Milestone:	---	Keywords:	Tracking
Target Release:	---
Hardware:	All
OS:	All
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-10-02 17:37:43 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	245418, 1269538, 1429132

Description Martin Kolman 2017-09-14 18:03:57 UTC

The Initial Setup utility which takes care for configuring an installed system is at it's core just a thin launcher for screens provided by the Anaconda installer.

For this reason, it makes sense for it to depend on the "anaconda-tui" and anaconda-gui packages (for the IS GUI package).

On the other hand, Initial Setup does now show *all* the Anaconda screens - it does not show the storage configuration screens, the installation source screen, the software selection screen, etc.

Yet it still transitively pulls in all dependencies even for the screen it does not use at all (eq. blivet & all the other storage stuff).

So it would make sense to modify the Anaconda package in such a way that Initial Setup can still use the screens it cares about (user creation, network configuration, etc.) while not pulling in all the unrelated dependencies.

This is even more important now due to the distribution modularization efforts.

Also other use-cases might benefit as well, for example Anaconda based image/container creation which also might not really require all the various storage related dependencies.

Comment 1 Peter Robinson 2018-02-08 17:55:36 UTC

Martin: what's the status of this? Is this blocking on any other changes as it would be really useful for atomic/arm minimal/IoT bits in F-28.

Comment 2 Martin Kolman 2018-02-09 00:10:03 UTC

(In reply to Peter Robinson from comment #1)
> Martin: what's the status of this? Is this blocking on any other changes as
> it would be really useful for atomic/arm minimal/IoT bits in F-28.

It actually turned out to be substantially more difficult than expected due to blivet being imported all over the place, to illustrate (trimming usage in tests and other false positives):


$ git grep "import blivet"
pyanaconda/anaconda.py:162:            import blivet.arch
pyanaconda/bootloader.py:24:import blivet
pyanaconda/display.py:40:import blivet
pyanaconda/exception.py:30:import blivet.errors
pyanaconda/image.py:25:import blivet.util
pyanaconda/image.py:26:import blivet.arch
pyanaconda/isys/__init__.py:28:import blivet.arch
pyanaconda/kickstart.py:31:import blivet.arch
pyanaconda/kickstart.py:32:import blivet.fcoe
pyanaconda/kickstart.py:33:import blivet.iscsi
pyanaconda/kickstart.py:34:import blivet.zfcp
pyanaconda/network.py:41:import blivet.arch
pyanaconda/payload/__init__.py:69:import blivet.util
pyanaconda/payload/__init__.py:70:import blivet.arch
pyanaconda/payload/livepayload.py:53:import blivet.util
pyanaconda/startup_utils.py:47:import blivet

$ git grep "from blivet"
pyanaconda/bootloader.py:32:from blivet.devicelibs import raid
pyanaconda/bootloader.py:33:from blivet.formats.disklabel import DiskLabel
pyanaconda/bootloader.py:36:from blivet.fcoe import fcoe
pyanaconda/bootloader.py:41:from blivet.size import Size
pyanaconda/bootloader.py:789:        from blivet.devices import NetworkStorageDevice
pyanaconda/format_dasd.py:25:from blivet import arch
pyanaconda/image.py:27:from blivet.errors import FSError
pyanaconda/installation.py:21:from blivet import callbacks
pyanaconda/installation.py:22:from blivet.devices import BTRFSDevice
pyanaconda/installclass.py:28:from blivet.size import Size
pyanaconda/installclasses/fedora_server.py:24:from blivet.size import Size
pyanaconda/kickstart.py:58:from blivet import udev
pyanaconda/kickstart.py:59:from blivet.deviceaction import ActionCreateFormat, ActionResizeDevice, ActionResizeFormat
pyanaconda/kickstart.py:60:from blivet.devicelibs.crypto import MIN_CREATE_ENTROPY
pyanaconda/kickstart.py:61:from blivet.devicelibs.lvm import LVM_PE_SIZE, KNOWN_THPOOL_PROFILES
pyanaconda/kickstart.py:62:from blivet.devices import LUKSDevice
pyanaconda/kickstart.py:63:from blivet.devices.lvm import LVMVolumeGroupDevice, LVMCacheRequest, LVMLogicalVolumeDevice
pyanaconda/kickstart.py:64:from blivet.errors import PartitioningError, StorageError, BTRFSValueError
pyanaconda/kickstart.py:65:from blivet.formats.disklabel import DiskLabel
pyanaconda/kickstart.py:66:from blivet.formats.fs import XFS
pyanaconda/kickstart.py:67:from blivet.formats import get_format
pyanaconda/kickstart.py:68:from blivet.partitioning import do_partitioning, grow_lvm
pyanaconda/kickstart.py:69:from blivet.size import Size, KiB
pyanaconda/network.py:40:from blivet.devices import FcoeDiskDevice
pyanaconda/payload/__init__.py:39:from blivet.size import Size, ROUND_HALF_UP
pyanaconda/payload/__init__.py:68:from blivet.errors import StorageError
pyanaconda/payload/dnfpayload.py:22:from blivet.size import Size
pyanaconda/payload/livepayload.py:52:from blivet.size import Size
pyanaconda/payload/rpmostreepayload.py:35:from blivet.size import Size
pyanaconda/payload/rpmostreepayload.py:36:from blivet.util import umount
pyanaconda/platform.py:25:from blivet import arch
pyanaconda/platform.py:26:from blivet.devicelibs import raid
pyanaconda/platform.py:27:from blivet.formats import get_device_format_class
pyanaconda/platform.py:28:from blivet.flags import flags
pyanaconda/platform.py:29:from blivet.size import Size
pyanaconda/rescue.py:19:from blivet.errors import StorageError
pyanaconda/rescue.py:20:from blivet.devices import LUKSDevice
pyanaconda/storage/autopart.py:26:from blivet import util
pyanaconda/storage/autopart.py:27:from blivet.size import Size
pyanaconda/storage/autopart.py:28:from blivet.devices.partition import PartitionDevice, FALLBACK_DEFAULT_PART_SIZE
pyanaconda/storage/autopart.py:29:from blivet.devices.luks import LUKSDevice
pyanaconda/storage/autopart.py:30:from blivet.devices.lvm import DEFAULT_THPOOL_RESERVE
pyanaconda/storage/autopart.py:31:from blivet.errors import NoDisksError, NotEnoughFreeSpaceError
pyanaconda/storage/autopart.py:32:from blivet.formats import get_format
pyanaconda/storage/autopart.py:33:from blivet.partitioning import do_partitioning, get_free_regions, grow_lvm
pyanaconda/storage/autopart.py:34:from blivet.static_data import luks_data
pyanaconda/storage/osinstall.py:37:from blivet import arch, udev
pyanaconda/storage/osinstall.py:38:from blivet import util as blivet_util
pyanaconda/storage/osinstall.py:39:from blivet.blivet import Blivet
pyanaconda/storage/osinstall.py:40:from blivet.storage_log import log_exception_info
pyanaconda/storage/osinstall.py:41:from blivet.devices import FileDevice, NFSDevice, NoDevice, OpticalDevice, NetworkStorageDevice, DirectoryDevice, MDRaidArrayDevice, PartitionDevice, BTRFSSubVolumeDevice, TmpFSDevice, LVMLogicalVolumeDevice, LVMVolumeGroupDevice, BTRFSDevice
pyanaconda/storage/osinstall.py:42:from blivet.errors import FSTabTypeMismatchError, UnrecognizedFSTabEntryError, StorageError, FSResizeError, FormatResizeError, UnknownSourceDeviceError
pyanaconda/storage/osinstall.py:43:from blivet.formats import get_device_format_class
pyanaconda/storage/osinstall.py:44:from blivet.formats import get_format
pyanaconda/storage/osinstall.py:45:from blivet.flags import flags as blivet_flags
pyanaconda/storage/osinstall.py:46:from blivet.iscsi import iscsi
pyanaconda/storage/osinstall.py:47:from blivet.fcoe import fcoe
pyanaconda/storage/osinstall.py:48:from blivet.zfcp import zfcp
pyanaconda/storage/osinstall.py:49:from blivet.size import Size
pyanaconda/storage/partspec.py:22:from blivet.util import stringize, unicodeize
pyanaconda/storage_utils.py:27:from blivet import arch
pyanaconda/storage_utils.py:28:from blivet import util
pyanaconda/storage_utils.py:29:from blivet import udev
pyanaconda/storage_utils.py:30:from blivet.size import Size
pyanaconda/storage_utils.py:31:from blivet.errors import StorageError
pyanaconda/storage_utils.py:32:from blivet.formats import device_formats
pyanaconda/storage_utils.py:33:from blivet.formats.fs import FS
pyanaconda/storage_utils.py:34:from blivet.devicefactory import DEVICE_TYPE_LVM
pyanaconda/storage_utils.py:35:from blivet.devicefactory import DEVICE_TYPE_LVM_THINP
pyanaconda/storage_utils.py:36:from blivet.devicefactory import DEVICE_TYPE_BTRFS
pyanaconda/storage_utils.py:37:from blivet.devicefactory import DEVICE_TYPE_MD
pyanaconda/storage_utils.py:38:from blivet.devicefactory import DEVICE_TYPE_PARTITION
pyanaconda/storage_utils.py:39:from blivet.devicefactory import DEVICE_TYPE_DISK
pyanaconda/timezone.py:34:from blivet import arch
pyanaconda/ui/gui/spokes/advanced_storage.py:27:from blivet import arch
pyanaconda/ui/gui/spokes/advanced_storage.py:28:from blivet.devices import DASDDevice, FcoeDiskDevice, iScsiDiskDevice, MultipathDevice, ZFCPDiskDevice
pyanaconda/ui/gui/spokes/advanced_storage.py:29:from blivet.fcoe import has_fcoe
pyanaconda/ui/gui/spokes/advanced_storage.py:30:from blivet.iscsi import iscsi
pyanaconda/ui/gui/spokes/advstorage/fcoe.py:20:from blivet.fcoe import fcoe
pyanaconda/ui/gui/spokes/advstorage/iscsi.py:33:from blivet.iscsi import iscsi
pyanaconda/ui/gui/spokes/advstorage/iscsi.py:34:from blivet.safe_dbus import SafeDBusError
pyanaconda/ui/gui/spokes/advstorage/zfcp.py:25:from blivet import zfcp
pyanaconda/ui/gui/spokes/blivet_gui.py:36:from blivetgui import osinstall
pyanaconda/ui/gui/spokes/blivet_gui.py:37:from blivetgui.config import config
pyanaconda/ui/gui/spokes/custom_storage.py:49:from blivet import devicefactory
pyanaconda/ui/gui/spokes/custom_storage.py:50:from blivet.formats import get_format
pyanaconda/ui/gui/spokes/custom_storage.py:51:from blivet.size import Size
pyanaconda/ui/gui/spokes/custom_storage.py:52:from blivet.devicefactory import DEVICE_TYPE_LVM
pyanaconda/ui/gui/spokes/custom_storage.py:53:from blivet.devicefactory import DEVICE_TYPE_BTRFS
pyanaconda/ui/gui/spokes/custom_storage.py:54:from blivet.devicefactory import DEVICE_TYPE_PARTITION
pyanaconda/ui/gui/spokes/custom_storage.py:55:from blivet.devicefactory import DEVICE_TYPE_MD
pyanaconda/ui/gui/spokes/custom_storage.py:56:from blivet.devicefactory import DEVICE_TYPE_DISK
pyanaconda/ui/gui/spokes/custom_storage.py:57:from blivet.devicefactory import DEVICE_TYPE_LVM_THINP
pyanaconda/ui/gui/spokes/custom_storage.py:58:from blivet.devicefactory import SIZE_POLICY_AUTO
pyanaconda/ui/gui/spokes/custom_storage.py:59:from blivet.devicefactory import is_supported_device_type
pyanaconda/ui/gui/spokes/custom_storage.py:60:from blivet.errors import StorageError
pyanaconda/ui/gui/spokes/custom_storage.py:61:from blivet.errors import NoDisksError
pyanaconda/ui/gui/spokes/custom_storage.py:62:from blivet.errors import NotEnoughFreeSpaceError
pyanaconda/ui/gui/spokes/custom_storage.py:63:from blivet.devicelibs import raid, crypto
pyanaconda/ui/gui/spokes/custom_storage.py:64:from blivet.devices import LUKSDevice, MDRaidArrayDevice, LVMVolumeGroupDevice
pyanaconda/ui/gui/spokes/installation_source.py:54:from blivet.util import get_mount_device, get_mount_paths
pyanaconda/ui/gui/spokes/lib/accordion.py:21:from blivet.devicefactory import is_supported_device_type
pyanaconda/ui/gui/spokes/lib/cart.py:23:from blivet.size import Size
pyanaconda/ui/gui/spokes/lib/custom_storage_helpers.py:43:from blivet.size import Size
pyanaconda/ui/gui/spokes/lib/custom_storage_helpers.py:44:from blivet.formats import get_format
pyanaconda/ui/gui/spokes/lib/custom_storage_helpers.py:45:from blivet.devicefactory import SIZE_POLICY_AUTO
pyanaconda/ui/gui/spokes/lib/custom_storage_helpers.py:46:from blivet.devicefactory import SIZE_POLICY_MAX
pyanaconda/ui/gui/spokes/lib/custom_storage_helpers.py:47:from blivet.devicefactory import DEVICE_TYPE_LVM
pyanaconda/ui/gui/spokes/lib/custom_storage_helpers.py:48:from blivet.devicefactory import DEVICE_TYPE_BTRFS
pyanaconda/ui/gui/spokes/lib/custom_storage_helpers.py:49:from blivet.devicefactory import DEVICE_TYPE_LVM_THINP
pyanaconda/ui/gui/spokes/lib/custom_storage_helpers.py:50:from blivet.devicefactory import DEVICE_TYPE_MD
pyanaconda/ui/gui/spokes/lib/custom_storage_helpers.py:51:from blivet.devicefactory import get_supported_raid_levels
pyanaconda/ui/gui/spokes/lib/custom_storage_helpers.py:52:from blivet.devicelibs import btrfs
pyanaconda/ui/gui/spokes/lib/custom_storage_helpers.py:53:from blivet.devicelibs import mdraid
pyanaconda/ui/gui/spokes/lib/custom_storage_helpers.py:54:from blivet.devicelibs import raid
pyanaconda/ui/gui/spokes/lib/entropy_dialog.py:33:from blivet.util import get_current_entropy
pyanaconda/ui/gui/spokes/lib/resize.py:32:from blivet.size import Size
pyanaconda/ui/gui/spokes/lib/resize.py:33:from blivet.formats.fs import FS
pyanaconda/ui/gui/spokes/lib/summary.py:24:from blivet.deviceaction import ACTION_TYPE_DESTROY, ACTION_TYPE_RESIZE, ACTION_OBJECT_FORMAT
pyanaconda/ui/gui/spokes/storage.py:63:from blivet.size import Size
pyanaconda/ui/gui/spokes/storage.py:64:from blivet.devices import MultipathDevice, ZFCPDiskDevice, iScsiDiskDevice
pyanaconda/ui/gui/spokes/storage.py:65:from blivet.errors import StorageError
pyanaconda/ui/gui/spokes/storage.py:66:from blivet.formats.disklabel import DiskLabel
pyanaconda/ui/gui/spokes/storage.py:67:from blivet.iscsi import iscsi
pyanaconda/ui/lib/disks.py:20:from blivet.devices import MultipathDevice, iScsiDiskDevice, FcoeDiskDevice
pyanaconda/ui/lib/entropy.py:35:from blivet.util import get_current_entropy
pyanaconda/ui/lib/space.py:20:from blivet.size import Size
pyanaconda/ui/tui/spokes/installation_source.py:41:from blivet.util import get_mount_device, get_mount_paths
pyanaconda/ui/tui/spokes/shell_spoke.py:27:from blivet import arch
pyanaconda/ui/tui/spokes/storage.py:29:from blivet.size import Size
pyanaconda/ui/tui/spokes/storage.py:30:from blivet.errors import StorageError
pyanaconda/ui/tui/spokes/storage.py:31:from blivet.devices import DASDDevice, FcoeDiskDevice, iScsiDiskDevice, MultipathDevice, ZFCPDiskDevice
pyanaconda/ui/tui/spokes/storage.py:32:from blivet.formats import get_format
scripts/anaconda-cleanup:41:from blivet.devicetree import DeviceTree

So while Initial Setup might not run the storage related screens, there are still many other places where Blivet gets imported that Initial Setup might hit either directly or via a (non storage related) screen. So even if we somehow adapted the packaging so that Blivet and it's dependencies are not installed, it would be enough to hit one of these places and there will be a traceback due to a missing module. We could do some mucking with on-demand imports of Blivet in function calls and/or be very careful what Initial Setup calls at runtime, but that really does not seem very robust to me.

I see two likely more complicated but potentially much more robust options:

1) Use week dependencies in Blivet packaging

Blivet itself is not a problem image size/dependencies wise, all the storage management stuff it pulls in is. Also it has a system for detecting if it actually has the tools it needs available and acting accordingly (basically all should be fine as long as you don't try to call stuff like partitioning, then it will start protesting). So Blivet could mark the storage management tools and libraries as a week dependencies, which could then be (if I'm understanding things correctly) be used to at compose of installation time to get Blivet installed, but without it's storage related dependencies. This way Initial Setup should be able to use Anaconda modules that import Blivet just fine without blowing up.

2) Split Blivet packaging

Alternatively a similar thing could be achieved by changing Blivet packaging to have a base package and another package that pulls in the storage dependencies. Then Anaconda would be made to depend only on the base package and the Blivet storage dependencies package would be only pulled in to installation images and live environments, where the storage management features are actually needed. This would not help only Initial Setup, but also directory installation scenarios that currently also pull in Blivet and it's deps.

On the other hand I don't know how doable this is without really knowing structure of Blivet in detail - adding Vojta Trefny to CC to possibly chime in.

3) Make use of the Anaconda modularization effort

We are currently working on an effort to turn Anaconda into a set of modules with a public DBUS interface[0][1]. This should among other things (stable addon API, better testing, rootless GUI, etc.) make it possibly to drop some of the modules when they are not needed. 

We plan to have a storage module, which will depend on Blivet and will likely contain all the Blivet calls. 

Then we could drop this module when running just Initial Setup, achieving the required dependency reduction.

As for timeframes - options 1 & 2 might theoretically be F28 material, but it would really depend on Blivet and it's maintainers and their capacity.

Option 3 is likely not F28 material - we are making good progress on the overall Anaconda Modularization effort, but the storage module will likely be one of the most (if not the most) complex modules.

Also - suggestions for alternative solutions are definitely welcome! :)

[0] https://fedoraproject.org/wiki/Changes/AnacondaModularization
[1] https://rhinstaller.wordpress.com/2017/10/09/anaconda-modularisation/

Comment 3 Peter Robinson 2018-02-21 10:29:35 UTC

(In reply to Martin Kolman from comment #2)
> (In reply to Peter Robinson from comment #1)
> > Martin: what's the status of this? Is this blocking on any other changes as
> > it would be really useful for atomic/arm minimal/IoT bits in F-28.
> 
> It actually turned out to be substantially more difficult than expected due
> to blivet being imported all over the place, to illustrate (trimming usage
> in tests and other false positives):

TBH I'm not overly surprised about this from poking at it in the past.

<snip grepped bits>

> So while Initial Setup might not run the storage related screens, there are
> still many other places where Blivet gets imported that Initial Setup might
> hit either directly or via a (non storage related) screen. So even if we
> somehow adapted the packaging so that Blivet and it's dependencies are not
> installed, it would be enough to hit one of these places and there will be a
> traceback due to a missing module. We could do some mucking with on-demand
> imports of Blivet in function calls and/or be very careful what Initial
> Setup calls at runtime, but that really does not seem very robust to me.

Yes, I don't see that hacking packaging just to remove deps is a proper solution 
to this, it's a hack and not one I wish to deal with all the support problems.

> I see two likely more complicated but potentially much more robust options:

I see 3 below ;-) although 1 and 2 basically look the same in slightly differnet 
form.

> 1) Use week dependencies in Blivet packaging
> 
> Blivet itself is not a problem image size/dependencies wise, all the storage
> management stuff it pulls in is. Also it has a system for detecting if it
> actually has the tools it needs available and acting accordingly (basically
> all should be fine as long as you don't try to call stuff like partitioning,
> then it will start protesting). So Blivet could mark the storage management
> tools and libraries as a week dependencies, which could then be (if I'm
> understanding things correctly) be used to at compose of installation time
> to get Blivet installed, but without it's storage related dependencies. This
> way Initial Setup should be able to use Anaconda modules that import Blivet
> just fine without blowing up.

This sounds like the packaging hack you mention above.

> 2) Split Blivet packaging
> 
> Alternatively a similar thing could be achieved by changing Blivet packaging
> to have a base package and another package that pulls in the storage
> dependencies. Then Anaconda would be made to depend only on the base package
> and the Blivet storage dependencies package would be only pulled in to
> installation images and live environments, where the storage management
> features are actually needed. This would not help only Initial Setup, but
> also directory installation scenarios that currently also pull in Blivet and
> it's deps.
> 
> On the other hand I don't know how doable this is without really knowing
> structure of Blivet in detail - adding Vojta Trefny to CC to possibly chime
> in.

This sounds like the packaging hack you mention above in a slightly different form.

> 3) Make use of the Anaconda modularization effort
> 
> We are currently working on an effort to turn Anaconda into a set of modules
> with a public DBUS interface[0][1]. This should among other things (stable
> addon API, better testing, rootless GUI, etc.) make it possibly to drop some
> of the modules when they are not needed. 

Yes, I'm aware of this.

> We plan to have a storage module, which will depend on Blivet and will
> likely contain all the Blivet calls. 
> 
> Then we could drop this module when running just Initial Setup, achieving
> the required dependency reduction.
> 
> As for timeframes - options 1 & 2 might theoretically be F28 material, but
> it would really depend on Blivet and it's maintainers and their capacity.

Personnally I see 1&2 as hacks, temporary dead ends.

> Option 3 is likely not F28 material - we are making good progress on the
> overall Anaconda Modularization effort, but the storage module will likely
> be one of the most (if not the most) complex modules.

<insert sad face>

> Also - suggestions for alternative solutions are definitely welcome! :)

The only two I have is:
1) import the bits of code needed by initial-setup directly into it and remove all
the bits, presumably the vast majority and have it as a standalone project.
2) Find another project to do the same thing and not use initial-setup at all


This is a blocker for two things:
1) Atomic Host on Single Board Computers such as the Raspberry Pi.
2) The Fedora IoT Objective.

The issue is it adds over 25% packages to the minimal image that is used once 
and basically never needed again in the lifecycle of the system. Without 
initial-setup we currently have around 350 packages in the minimal image, add 
initial-setup and that's up over 450 adding a large dep tree which causes problems
in both updates volume/delta and attack surface from a security PoV.


See the example below. This isn't 100% accurate as dnf is removes a handful of 
packages that we'd actually want but it shows fairly accurately the problem we 
have, and with atomic you just can't do "dnf remove"


# dnf remove initial-setup
Dependencies resolved.
================================================================================
 Package                         Arch    Version                 Repository
                                                                           Size
================================================================================
Removing:
 initial-setup                   armv7hl 0.3.53-2.fc28           @rawhide 171 k
Removing dependent packages:
 anaconda-core                   armv7hl 28.22-1.fc28            @rawhide 9.3 M
 anaconda-tui                    armv7hl 28.22-1.fc28            @rawhide 363 k
 dmraid                          armv7hl 1.0.0.rc16-37.fc28      @rawhide 326 k
 dmraid-events                   armv7hl 1.0.0.rc16-37.fc28      @rawhide  12 k
 iscsi-initiator-utils-iscsiuio  armv7hl 6.2.0.874-9.git86e8892.fc28
                                                                 @rawhide 157 k
 kexec-tools                     armv7hl 2.0.16-5.fc28           @rawhide 313 k
 langtable-data                  noarch  0.0.38-3.fc28           @rawhide 709 k
 langtable-python3               noarch  0.0.38-3.fc28           @rawhide 125 k
 libblockdev-btrfs               armv7hl 2.16-2.fc28             @rawhide  19 k
 libblockdev-crypto              armv7hl 2.16-2.fc28             @rawhide  23 k
 libblockdev-dm                  armv7hl 2.16-2.fc28             @rawhide  19 k
 libblockdev-fs                  armv7hl 2.16-2.fc28             @rawhide  50 k
 libblockdev-kbd                 armv7hl 2.16-2.fc28             @rawhide  31 k
 libblockdev-lvm                 armv7hl 2.16-2.fc28             @rawhide  40 k
 libblockdev-mpath               armv7hl 2.16-2.fc28             @rawhide  15 k
 libblockdev-part                armv7hl 2.16-2.fc28             @rawhide  35 k
 libblockdev-plugins-all         armv7hl 2.16-2.fc28             @rawhide   0
 libproxy                        armv7hl 0.4.15-5.fc28           @rawhide 129 k
 libreport                       armv7hl 2.9.3-6.fc28            @rawhide 2.3 M
 libreport-anaconda              armv7hl 2.9.3-6.fc28            @rawhide  18 k
 libreport-cli                   armv7hl 2.9.3-6.fc28            @rawhide  29 k
 libreport-plugin-bugzilla       armv7hl 2.9.3-6.fc28            @rawhide 143 k
 libreport-plugin-reportuploader armv7hl 2.9.3-6.fc28            @rawhide  86 k
 libreport-web                   armv7hl 2.9.3-6.fc28            @rawhide  40 k
 libteam                         armv7hl 1.27-6.fc28             @rawhide  82 k
 lvm2                            armv7hl 2.02.177-4.fc28         @rawhide 2.8 M
 lvm2-libs                       armv7hl 2.02.177-4.fc28         @rawhide 3.3 M
 python-systemd-doc              armv7hl 234-5.fc28              @rawhide 753 k
 python3-blivet                  noarch  1:3.0.0-0.2.b1.fc28     @rawhide 3.0 M
 python3-blockdev                armv7hl 2.16-2.fc28             @rawhide  64 k
 python3-bytesize                armv7hl 1.2-3.fc28              @rawhide  21 k
 python3-cryptography            armv7hl 2.1.4-1.fc28            @rawhide 2.3 M
 python3-iscsi-initiator-utils   armv7hl 6.2.0.874-9.git86e8892.fc28
                                                                 @rawhide  20 k
 python3-kickstart               noarch  3.12-1.fc28             @rawhide 1.7 M
 python3-libreport               armv7hl 2.9.3-6.fc28            @rawhide 102 k
 python3-meh                     noarch  0.46-2.fc28             @rawhide 277 k
 python3-pyOpenSSL               noarch  17.3.0-3.fc28           @rawhide 509 k
 python3-requests                noarch  2.18.4-3.fc28           @rawhide 410 k
 python3-requests-file           noarch  1.4.3-4.fc28            @rawhide 9.4 k
 python3-requests-ftp            noarch  0.3.1-9.fc28            @rawhide  33 k
 python3-urllib3                 noarch  1.22-6.fc28             @rawhide 629 k
 teamd                           armv7hl 1.27-6.fc28             @rawhide 241 k
 volume_key-libs                 armv7hl 0.3.9-19.fc28           @rawhide 670 k
 xmlrpc-c-client                 armv7hl 1.51.0-4.fc28           @rawhide  40 k
Removing unused dependencies:
 NetworkManager-team             armv7hl 1:1.10.2-1.fc28         @rawhide  54 k
 augeas-libs                     armv7hl 1.10.1-2.fc28           @rawhide 1.1 M
 bcache-tools                    armv7hl 1.0.8-11.fc28           @rawhide  84 k
 blivet-data                     noarch  1:3.0.0-0.2.b1.fc28     @rawhide 356 k
 btrfs-progs                     armv7hl 4.15.1-1.fc28           @rawhide 4.6 M
 createrepo_c                    armv7hl 0.10.0-18.fc28          @rawhide 128 k
 createrepo_c-libs               armv7hl 0.10.0-18.fc28          @rawhide 192 k
 device-mapper-event             armv7hl 1.02.146-4.fc28         @rawhide  37 k
 device-mapper-event-libs        armv7hl 1.02.146-4.fc28         @rawhide  49 k
 device-mapper-multipath         armv7hl 0.7.4-1.git07e7bd5.fc28 @rawhide 204 k
 device-mapper-multipath-libs    armv7hl 0.7.4-1.git07e7bd5.fc28 @rawhide 539 k
 device-mapper-persistent-data   armv7hl 0.7.5-3.fc28            @rawhide 1.2 M
 dracut-network                  armv7hl 047-1.fc28              @rawhide 149 k
 drpm                            armv7hl 0.3.0-13.fc28           @rawhide 130 k
 ethtool                         armv7hl 2:4.15-1.fc28           @rawhide 363 k
 fedora-logos                    armv7hl 28.0.2-3.fc28           @rawhide 8.9 M
 gdisk                           armv7hl 1.0.3-5.fc28            @rawhide 723 k
 gnupg                           armv7hl 1.4.22-6.fc28           @rawhide 5.3 M
 iscsi-initiator-utils           armv7hl 6.2.0.874-9.git86e8892.fc28
                                                                 @rawhide 1.4 M
 isns-utils-libs                 armv7hl 0.97-6.fc28             @rawhide 322 k
 isomd5sum                       armv7hl 1:1.2.2-2.fc28          @rawhide  59 k
 js-jquery                       noarch  3.2.1-3.fc28            @rawhide 508 k
 langtable                       noarch  0.0.38-3.fc28           @rawhide 121 k
 libaio                          armv7hl 0.3.110-11.fc28         @rawhide  37 k
 libblockdev                     armv7hl 2.16-2.fc28             @rawhide 243 k
 libblockdev-loop                armv7hl 2.16-2.fc28             @rawhide  10 k
 libblockdev-mdraid              armv7hl 2.16-2.fc28             @rawhide  23 k
 libblockdev-swap                armv7hl 2.16-2.fc28             @rawhide  15 k
 libblockdev-utils               armv7hl 2.16-2.fc28             @rawhide  30 k
 libbytesize                     armv7hl 1.2-3.fc28              @rawhide 106 k
 libdaemon                       armv7hl 0.14-15.fc28            @rawhide  57 k
 libmodman                       armv7hl 2.0.1-17.fc28           @rawhide  53 k
 libnl3-cli                      armv7hl 3.4.0-3.fc28            @rawhide 600 k
 libtar                          armv7hl 1.2.20-12.fc28          @rawhide  66 k
 libusb                          armv7hl 1:0.1.5-12.fc28         @rawhide  62 k
 lsof                            armv7hl 4.89-8.fc28             @rawhide 625 k
 lz4                             armv7hl 1.8.1.2-4.fc28          @rawhide 167 k
 lzo                             armv7hl 2.08-12.fc28            @rawhide 163 k
 mdadm                           armv7hl 4.0-5.fc27              @koji-override-0
 python3-asn1crypto              noarch  0.23.0-2.fc28           @rawhide 835 k
 python3-cffi                    armv7hl 1.11.2-1.fc28           @rawhide 960 k
 python3-chardet                 noarch  3.0.4-5.fc28            @rawhide 903 k
 python3-coverage                armv7hl 4.5.1-1.fc28            @rawhide 936 k
 python3-idna                    noarch  2.5-4.fc28              @rawhide 509 k
 python3-ntplib                  noarch  0.3.3-9.fc28            @rawhide  28 k
 python3-ordered-set             noarch  2.0.2-3.fc28            @rawhide  16 k
 python3-pid                     noarch  2.1.1-6.fc28            @rawhide  33 k
 python3-pwquality               armv7hl 1.4.0-6.fc28            @rawhide  15 k
 python3-pycparser               noarch  2.14-13.fc28            @rawhide 810 k
 python3-pydbus                  noarch  0.6.0-5.fc28            @rawhide 122 k
 python3-pyparted                armv7hl 1:3.11.0-9.fc28         @rawhide 329 k
 python3-pysocks                 noarch  1.6.8-2.fc28            @rawhide  75 k
 python3-pytz                    noarch  2017.2-4.fc28           @rawhide 174 k
 python3-pyudev                  noarch  0.21.0-5.fc28           @rawhide 315 k
 python3-simpleline              noarch  0.8-2.fc28              @rawhide 378 k
 python3-systemd                 armv7hl 234-5.fc28              @rawhide 237 k
 python3-wrapt                   armv7hl 1.10.11-2.fc28          @rawhide 145 k
 realmd                          armv7hl 0.16.3-9.fc28           @rawhide 702 k
 rsync                           armv7hl 3.1.3-3.fc28            @rawhide 811 k
 satyr                           armv7hl 0.25-2.fc28             @rawhide 252 k
 sgpio                           armv7hl 1.2.0.10-20.fc28        @rawhide  18 k
 userspace-rcu                   armv7hl 0.10.1-2.fc28           @rawhide 284 k
 web-assets-filesystem           noarch  5-7.fc28                @rawhide   0
 xmlrpc-c                        armv7hl 1.51.0-4.fc28           @rawhide 487 k

Transaction Summary
================================================================================
Remove  109 Packages

Freed space: 70 M 
Is this ok [y/N]:

Comment 4 Peter Robinson 2018-02-21 10:37:45 UTC

> The issue is it adds over 25% packages to the minimal image that is used once 
> and basically never needed again in the lifecycle of the system. Without 
> initial-setup we currently have around 350 packages in the minimal image, add 
> initial-setup and that's up over 450 adding a large dep tree which causes problems
> in both updates volume/delta and attack surface from a security PoV.

In fact this system has packages installed on top of minimal so it's in fact slightly larger percentage.

Comment 5 Martin Kolman 2018-02-21 11:54:27 UTC

<snip>
> The only two I have is:
> 1) import the bits of code needed by initial-setup directly into it and
> remove all
> the bits, presumably the vast majority and have it as a standalone project.
This is actually how it was before with the Firstboot tool - and it was a huge mess. There was basically another implementation of some of the Anaconda screens, which very quickly got out of sync and was getting significantly less testing and user feedback. This resulted in a tool that was pretty buggy while being a significant maintenance burden. 

That was the reason why Initial Setup was created - to just be a lightweight launcher for Anaconda spokes, reducing the maintenance burden and using common tested code for screens.

<snip>
>This is a blocker for two things:
>1) Atomic Host on Single Board Computers such as the Raspberry Pi.
>2) The Fedora IoT Objective.
That's a good point - I'll try to look into this a bit more and/or accelerate the relevant Anaconda modularity work.

Comment 6 Peter Robinson 2018-02-21 11:57:34 UTC

(In reply to Martin Kolman from comment #5)
> <snip>
> > The only two I have is:
> > 1) import the bits of code needed by initial-setup directly into it and
> > remove all
> > the bits, presumably the vast majority and have it as a standalone project.
> This is actually how it was before with the Firstboot tool - and it was a
> huge mess. There was basically another implementation of some of the
> Anaconda screens, which very quickly got out of sync and was getting
> significantly less testing and user feedback. This resulted in a tool that
> was pretty buggy while being a significant maintenance burden. 
> 
> That was the reason why Initial Setup was created - to just be a lightweight
> launcher for Anaconda spokes, reducing the maintenance burden and using
> common tested code for screens.

I realise that, I'm not saying the idea was a good one... ;-)

Comment 7 Peter Robinson 2018-02-21 12:32:51 UTC

There's also issues like the "fix" for rhbz #1529239 "Move system-logos to anaconda-core" where due to an issue in syslinux the dependency was moved to the core package where syslinux isn't even the default bootloader on x86, and isn't supported on any other platform.

The fix for that should have been in lorax, likely here:
https://github.com/rhinstaller/lorax/blob/master/share/templates.d/99-generic/runtime-install.tmpl#L53

Comment 8 Peter Robinson 2018-02-21 12:45:28 UTC

(In reply to Peter Robinson from comment #7)
> There's also issues like the "fix" for rhbz #1529239 "Move system-logos to
> anaconda-core" where due to an issue in syslinux the dependency was moved to
> the core package where syslinux isn't even the default bootloader on x86,
> and isn't supported on any other platform.
> 
> The fix for that should have been in lorax, likely here:
> https://github.com/rhinstaller/lorax/blob/master/share/templates.d/99-
> generic/runtime-install.tmpl#L53

https://github.com/rhinstaller/lorax/pull/327

Comment 9 Martin Kolman 2018-02-21 14:00:30 UTC

(In reply to Peter Robinson from comment #8)
> (In reply to Peter Robinson from comment #7)
> > There's also issues like the "fix" for rhbz #1529239 "Move system-logos to
> > anaconda-core" where due to an issue in syslinux the dependency was moved to
> > the core package where syslinux isn't even the default bootloader on x86,
> > and isn't supported on any other platform.
> > 
> > The fix for that should have been in lorax, likely here:
> > https://github.com/rhinstaller/lorax/blob/master/share/templates.d/99-
> > generic/runtime-install.tmpl#L53
> 
> https://github.com/rhinstaller/lorax/pull/327

Nice! Anaconda PR dropping the dependency: https://github.com/rhinstaller/anaconda/pull/1347

Comment 10 Peter Robinson 2018-02-21 14:25:51 UTC

> Nice! Anaconda PR dropping the dependency:
> https://github.com/rhinstaller/anaconda/pull/1347

Actually I think that's wrong, I'm sure the logos are used in the GUI too, so it should be just moved back to the -gui sub package. See the change in the anaconda package here:
 
https://src.fedoraproject.org/rpms/anaconda/c/3fd81384395a225480adaa1b0a51d6cf94a0286e?branch=master

Comment 11 Martin Kolman 2018-02-21 14:42:22 UTC

(In reply to Peter Robinson from comment #10)
> > Nice! Anaconda PR dropping the dependency:
> > https://github.com/rhinstaller/anaconda/pull/1347
> 
> Actually I think that's wrong, I'm sure the logos are used in the GUI too,
> so it should be just moved back to the -gui sub package. See the change in
> the anaconda package here:
>  
> https://src.fedoraproject.org/rpms/anaconda/c/
> 3fd81384395a225480adaa1b0a51d6cf94a0286e?branch=master

Oh good point - if I understand things correctly, we actually have two issues:
1) the Anaconda GUI needs system-logos to show logos, so anaconda-gui should depend on system-logos
2) some non-GUI installation-environment-only tools need system-logos as well

So 1) should be fixed by moving the dependency to anaconda-gui (I'll update the PR) and 2) by the Lorax template change, right ?

Comment 12 Peter Robinson 2018-02-21 15:07:16 UTC

> Oh good point - if I understand things correctly, we actually have two
> issues:
> 1) the Anaconda GUI needs system-logos to show logos, so anaconda-gui should
> depend on system-logos

Correct, and it use to until it was moved to core to deal, IMO incorrectly, with the issue below.

> 2) some non-GUI installation-environment-only tools need system-logos as well

Not exactly, lorax uses the logo for syslinux config creation on x86 arches when that bootloader is being using (live images I think) to create a pretty boot screen. See this template:
https://github.com/rhinstaller/lorax/blob/master/share/templates.d/99-generic/x86.tmpl#L44

> So 1) should be fixed by moving the dependency to anaconda-gui (I'll update
> the PR) and 2) by the Lorax template change, right ?

Correct!

Comment 13 Peter Robinson 2018-02-27 17:57:27 UTC

So I had a, possibly crazy, idea that maybe these deps could be handled with sub packaged rather than code changes, basically move all the bits in core/gui that aren't needed by initial-setup to say anaconda-core-installer and anaconda-core-installer-gui packages, with the standalone bits (user, timezone, network etc) in core and gui, and have the -installer packages depend on those, and then the root anaconda depend on it too, so there shouldn't be any dep changes but the bits that include the full blivet and other deps chain aren't pulled in by initial setup, should be no change for anaconda installer.

Thoughts?

Comment 14 Martin Kolman 2018-04-05 14:12:50 UTC

So in the end we decided to create an Anaconda metapackage holding all the installation environment only dependencies. The other options we have investigated have been moving the deps to the Lorax template or a comps group.

We went with the metapackage as it makes it possible to finally consolidate all the Anaconda dependencies to be tracked in a single place (the Anaconda spec file).

The Anaconda PR is here:
https://github.com/rhinstaller/anaconda/pull/1415
With related PRs for Lorax and Fedora kickstarts:
https://github.com/weldr/lorax/pull/347
https://pagure.io/fedora-kickstarts/pull-request/375

I've already moved some installation environment includes from the Lorax template to the metapackage and likely the same thing will happen for the anaconda-tools comps group in the future. Then we will finally have *all* the install time dependencies listed in a single place.

And as Initial Setup will not depend on the anaconda-install-env-deps package, it will be spared from pulling all those in.

So getting back to the original issue of this bug (reduction in the amount of unnecessary dependencies Initial Setup pulls in) a lot should be achieved once the PRs mentioned above are merged.

There is also a related discussion on the Blivet side, about Blivet creating a similar storage-dependencies metapackage, so the applications can depend on Blivet for things like architecture detection or storage modeling without pulling all the storage handling stuff.

Comment 15 Martin Kolman 2019-10-02 17:37:43 UTC

This has been effectively implemented long ago, so lets finally close the bug.