sgallagh, is there anything we can do to improve communication here to ensure that changes like these get made in PackageKit as well as dnf? (And probably dnfdragora, too; I'm about to go test what happens if you install dwm in Rawhide with dnfdragora...)

dnfdragora actually works OK, I think because dnfdaemon actually winds up using the dnf `resolve()` method where this is done in dnf.

Proposed as a Freeze Exception for 29-final by Fedora user zbyszek using the blocker tracking app because:

 @adawm wrote:
> Proposing as a Fedora 30 Beta blocker per criterion "The installed system must be able appropriately to install, remove, and update software with the default console tool for the relevant software type (e.g. default console package manager). This includes downloading of packages to be installed/updated" - for packages in default module streams I would say this constitutes a violation of "appropriately", there. Not filing for F29 as I believe we are not using default module streams for F29.

But that last sentence is not correct: we now have "default module streams" in F29 (even if just two).

Note, see also https://bugzilla.redhat.com/show_bug.cgi?id=1636184 , which was proposed as a blocker but 'fixed' before it could be voted on. It now seems that while *that* default module stream was removed, two others still exist (dwm and stratis).

Discussed during the 2018-10-15 blocker review meeting: [1]

The decision to classify this bug as an "AcceptedBlocker" was made as default module streams appearing in F29 was not expected by all parties involved. We agree at least that this bug must be fixed *OR* default module streams must be delayed to Fedora 30. We will also file a FESCo ticket to ensure FESCo is aware of this situation.

[1] https://meetbot.fedoraproject.org/fedora-blocker-review/2018-10-15/f29-blocker-review.2018-10-15-16.00.txt

Just to note: upgrading to libdnf 0.22.0 does *not* fix this issue.

CCing Matt just for info here, regarding the whole 'default module streams were a surprise to some of us' angle. There's an extended discussion in the full meeting log.

This isn't possible to fix in PackageKit. The only way to trigger moduleContainer->save() seems to be from dnf_context_run(), which PK can't use as it needs more control over when packages are downloaded. In C it's possible to call dnf_context_get_sack() to get the context sack, but dnf_sack_get_module_container() isn't exported, and ModulePackageContainer isn't available from C. We can't switch to dnf_context_run() as we need to run dnf_transaction_depsolve() as a unique step so we know what permissions are required before choosing an appropriate PolicyKit action.

If moduleContainer->save() is moved from dnf_context_run() to dnf_transaction_commit() then it magically will start working for PackageKit with no changes to the PackageKit backend required. It also doesn't require exporting any more API if it's solved this way. Kalev agrees, and added "commit seems like semantically better place for this anyway" which I agree with.

I create PR that should solve the issue (https://github.com/rpm-software-management/libdnf/pull/619). But before merge we need an explicit confirmation from  Richard Hughes that the solution works with PackageKit. Please Richard can you test it to ensure that it works correctly?

The patch is part of libdnf-0.22.0-4.fc29, but I cannot edit bodhi update https://bodhi.fedoraproject.org/updates/FEDORA-2018-2789f6b6e7 to get it into Fedora 29. Please can anyone help me?

(In reply to Jaroslav Mracek from comment #10)
> The patch is part of libdnf-0.22.0-4.fc29, but I cannot edit bodhi update
> https://bodhi.fedoraproject.org/updates/FEDORA-2018-2789f6b6e7 to get it
> into Fedora 29. Please can anyone help me?

Any provenpacager can edit the update but you could also file a new update that will obsolete this one.

anaconda-29.24.7-1.fc29 dnf-4.0.4-1.fc29 dnf-plugins-core-4.0.0-2.fc29 libblockdev-2.20-2.fc29 libdnf-0.22.0-4.fc29 has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2018-2789f6b6e7

This crashes on me when run in a PackageKit context:

--- 8< ---
root@gibraltar:~> pkcon install dwm
Resolving                     [=========================]         
Loading cache                 [=========================]         
Testing changes               [=========================]         
Finished                      [                         ] (0%)  
The following packages have to be installed:
 dmenu-4.8-2.fc29.x86_64	Generic menu for X
 dwm-6.1-8.module_1995+c3e93812.x86_64	Dynamic window manager for X
 st-0.8.1-3.fc29.x86_64	A simple terminal implementation for X
Proceed with changes? [N/y] y

                              [=========================]         
Installing                    [=========================]         
Querying                      [=========================]         
Downloading packages          [=========================]         
Testing changes               [=========================]         
Installing packages           [======================== ] (99%)  The daemon crashed mid-transaction!
root@gibraltar:~>
--- >8 ---

It crashes because the sack in the DNF context isn't initialized (this is with a debugging build of libdnf):

--- 8< ---
(gdb) bt
#0  0x00007fffe830bcf1 in dnf_sack_get_module_container(_DnfSack*) (sack=0x0)
    at /home/nils/src/libdnf/libdnf/dnf-sack.cpp:2018
#1  0x00007fffe8326c25 in dnf_transaction_commit(DnfTransaction*, HyGoal, DnfState*, GError**) (transaction=0x5555555ccab0, 
    goal=0x55555592a9d0, state=0x7fffd4001910, error=0x7fffda4f7cd0)
    at /home/nils/src/libdnf/libdnf/dnf-transaction.cpp:1440
#2  0x00007ffff7fc0d90 in pk_backend_transaction_download_commit
    (error=0x7fffda4f7cd0, state=0x7fffd40019f0, job=<optimized out>)
    at pk-backend-dnf.c:2363
--- >8 ---

How it is possible after testing?

I create an additional patch for PackageKit (https://github.com/rpm-software-management/libdnf/pull/620), that should solve the issue from comment 13. Again before merge we need an explicit confirmation from  Richard Hughes that the solution works with PackageKit. Please Richard can you test it to ensure that it works correctly?

anaconda-29.24.7-1.fc29, dnf-4.0.4-1.fc29, dnf-plugins-core-4.0.0-2.fc29, libblockdev-2.20-2.fc29, libdnf-0.22.0-5.fc29 has been pushed to the Fedora 29 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-2789f6b6e7

Unfortunately, this fixes just the crash but doesn't enable the module for me:

root@gibraltar:~> pkcon -y install dwm
Resolving                     [=========================]         
Installing                    [=========================]         
Loading cache                 [=========================]         
Downloading repository information[=========================]         
Loading cache                 [=========================]         
Downloading repository information[=========================]         
Loading cache                 [=========================]         
Downloading packages          [=========================]         
Testing changes               [=========================]         
Installing packages           [=========================]         
Finished                      [=========================]         
root@gibraltar:~> dnf module list --enabled
Copr repo for pulseaudio-dlna owned by cygn     173  B/s | 345  B     00:01    
RCM Tools for Fedora 29 (RPMs)                  0.0  B/s |   0  B     00:01    
Failed to synchronize cache for repo 'cygn-pulseaudio-dlna', ignoring this repo.
Failed to synchronize cache for repo 'rcm-tools-fedora-rpms', ignoring this repo.
Last metadata expiration check: 0:15:07 ago on Sat 20 Oct 2018 22:03:40 CEST.
Fedora Modular 29 - x86_64
Name     Stream     Profiles                                Summary             
gimp     2.10 [e]   devel [i], default [i]                  GIMP                
nodejs   8 [e]      development, minimal, default [d] [i]   Javascript runtime  

Hint: [d]efault, [e]nabled, [x]disabled, [i]nstalled
root@gibraltar:~>

This is with a manually started packagekitd from the Fedora package, against a local build of libdnf with this patch.

I'm not too familiar with either code bases but I think the sack must be present in order for the module to be enabled to be recorded.

Just a shot in the dark, but would this involve a call to dnf_context_setup_sack() in the right place?

It looks like that the problem is little bit more complex and should be solved in cooperation with PackageKit (probably changes in PackageKit required).

I update a patch for PackageKit (https://github.com/rpm-software-management/libdnf/pull/620), that should solve the issue from comment 13. Again before merge we need an explicit confirmation from  Richard Hughes that the solution works with PackageKit. Please Richard can you test it to ensure that it works correctly?

It uses sack from goal, therefore it should be present.

https://github.com/rpm-software-management/libdnf/pull/620#issuecomment-431842106

TL;DR: this updated patch works in my tests. Thanks!

anaconda-29.24.7-1.fc29, dnf-4.0.4-1.fc29, dnf-plugins-core-4.0.0-2.fc29, libblockdev-2.20-2.fc29, libdnf-0.22.0-5.fc29 has been pushed to the Fedora 29 stable repository. If problems still persist, please make note of it in this bug report.

libdnf-0.22.0-6.fc29 has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2018-5a90628a95

This version of libdnf works as expected. If I install a module using packagekit, the stream gets automatically enabled. Verified.

libdnf-0.22.0-6.fc29 has been pushed to the Fedora 29 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-5a90628a95

libdnf-0.22.0-6.fc29 has been pushed to the Fedora 29 stable repository. If problems still persist, please make note of it in this bug report.