Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1205649 - virtualbox/nvidia driver installed makes newer kernel updates to be ignored in packagekit
Summary: virtualbox/nvidia driver installed makes newer kernel updates to be ignored i...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: PackageKit
Version: 22
Hardware: x86_64
OS: Linux
unspecified
unspecified
Target Milestone: ---
Assignee: Richard Hughes
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: RejectedBlocker https://fedoraproject...
: 1204533 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-03-25 12:07 UTC by Lukas Brabec
Modified: 2015-12-12 15:05 UTC (History)
13 users (show)

Fixed In Version: PackageKit-1.0.6-6.fc22
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-06-21 00:06:35 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
reproducer script (deleted)
2015-03-25 12:07 UTC, Lukas Brabec
no flags Details
Fedora 22 reproducer (deleted)
2015-03-25 13:38 UTC, Lukas Brabec
no flags Details
packagekitd --verbose log (deleted)
2015-03-26 09:02 UTC, Lukas Brabec
no flags Details
packagekitd --verbose (with nvidia drivers) (deleted)
2015-03-26 12:34 UTC, Lukas Brabec
no flags Details
Fedora 22 reproducer version 2 (deleted)
2015-06-08 11:24 UTC, Kamil Páral
no flags Details

Description Lukas Brabec 2015-03-25 12:07:11 UTC
Created attachment 1006301 [details]
reproducer script

Description of problem:
(using rpmfusion repos)
VirtualBox needs kernel modules to run. These modules are installed via meta package kmod-VirtualBox, which has kmod-VirtualBox-<kernelversion> (kernel version is part of the name) as dependency and this dependent package requires given kernel version.

When three kernels are present on system (along with VirtualBox modules) and there is a kernel update, the oldest kernel (if not currently running) gets removed (and corresponding kmod-VirtualBox-<kernelversion> too). This is expected behavior and it is exactly what 'yum update' does.

However, offline updates, which uses 'pkcon get-updates' (triggered from console or from gnome software center) ignores the kernel update for some reason. Everything gets updated, but kernel.

Having VirtualBox installed and updating via offline updates (e.g. gnome software center) means you never receive newer kernels, but stay on the old ones all the time. That means you don't receive even security updates of the kernel.


Version-Release number of selected component (if applicable):
Fedora 21 Live
PackageKit-1.0.4-1.fc21.x86_64
gnome-packagekit-3.14.2-1.fc21.x86_64
libsolv-0.6.4-3.fc21.x86_64
libhif-0.1.8-4.fc21.x86_64
librepo-1.7.13-1.fc21.x86_64
yum-3.4.3-153.fc21.noarch
gnome-software-3.14.3-1.fc21.x86_64


I created shell script mimicking the VirtualBox packages to reproduce this on F21 Live installation. Its attached here. Simulating this with VirtualBox packages is much harder to reproduce, that's why I used fake packages to show this issue.

Steps to Reproduce:

1. install Fedora F21 Live
2. boot it and download the reproducer script mentioned
3. run the script as root (and have a coffee since it's updating the system)
4. reboot (make sure to boot the latest kernel)
5. pkcon refresh force
6. compare output of the following: 
     pkcon get-updates
     yum update
     gnome-software (Updates tab)
7. to-be-updated packages are different, yum wants to update the kernel, but pkcon (and gnome-software) does not


Example of 'pkcon get-updates' output:
Getting updates               [=========================]         
Loading cache                 [=========================]         
Finished                      [=========================]         
Available   	devassistant-0.9.3-4.fc21.noarch (updates)                  	DevAssistant - Making life easier for developers
Available   	kmod-foo-0.1-2.x86_64 (foo)                                 	Dummy summary


Example of 'yum update' output:
================================================================================
 Package         Arch    Version            Repository                     Size
================================================================================
Installing:
 kernel          x86_64  3.19.1-201.fc21    updates                        48 k
 kernel-core     x86_64  3.19.1-201.fc21    updates                        19 M
 kernel-modules  x86_64  3.19.1-201.fc21    updates                        17 M
Updating:
 devassistant    noarch  0.9.3-4.fc21       updates                       306 k
 kmod-foo        x86_64  0.1-2              foo                           5.6 k
Removing:
 kernel          x86_64  3.17.4-301.fc21    @koji-override-0/$releasever  0.0  
 kernel-core     x86_64  3.17.4-301.fc21    @koji-override-0/$releasever   40 M
 kernel-modules  x86_64  3.17.4-301.fc21    @koji-override-0/$releasever   17 M
Installing for dependencies:
 foo-3.19.1-201  x86_64  0.1-2              foo                           5.7 k
Removing for dependencies:
 foo-3.17.4-301  x86_64  0.1-1              @foo                          0.0

Comment 1 Kamil Páral 2015-03-25 12:18:05 UTC
I can confirm this issue, I've seen this for weeks on my F21 home laptop. I have VirtualBox installed there, and offline upgrades never upgrade my kernel (nor virtualbox modules), I always have to do it manually by using yum. This has been happening every week for the past ~two months.

I asked Lukas to reproduce this with rpmfluff-made packages, because reproducing this with real virtualbox packages is difficult - it depends on the current state of two repos (fedora updates and rpmfusion updates) and sometimes they are not in sync. Rpmfluff allowed us to make the reproducer easy. There's one tiny difference though, this line in reproducer should use '=' instead of '>=' to mimick the virtualbox behavior exactly:

foo2.add_requires("kernel-uname-r >= 3.19.1-201.fc21.x86_64")

However, because by the time you try this, there can be a newer kernel update in updates repo, we settled for '>=' to make sure this reproducer still works in the future.

Comment 2 Kalev Lember 2015-03-25 12:57:01 UTC
Does DNF handle it correctly?

Comment 3 Lukas Brabec 2015-03-25 13:34:33 UTC
Yep, DNF handles it correctly.

Comment 4 Lukas Brabec 2015-03-25 13:38:32 UTC
Created attachment 1006336 [details]
Fedora 22 reproducer

Comment 5 Lukas Brabec 2015-03-25 13:43:30 UTC
I was able to reproduce this on Fedora 22 installed from 22 Alpha RC3 Live Workstation. Steps are the same as above, just use the 22 reproducer I attached.

Version-Release number of selected component (if applicable):
Fedora 22 Live Alpha RC3
PackageKit-1.0.5-1.fc22.x86_64
librepo-1.7.13-1.fc22.x86_64
libhif-0.1.8-5.fc22.x86_64
librepo-1.7.13-1.fc22.x86_64
yum-3.4.3-155.fc22.noarch
gnome-software-3.14.3-1.fc21.x86_64

Comment 6 Fedora Blocker Bugs Application 2015-03-25 13:48:20 UTC
Proposed as a Blocker for 22-final by Fedora user lbrabec using the blocker tracking app because:

 I propose this as final blocker as it violates the (beta) criterion: 
Updates
The installed system must be able to download and install updates with the default graphical package manager in all release-blocking desktops.

Although its beta criterion, I propose this as final blocker - it is affected by only one package.

Comment 7 Rex Dieter 2015-03-25 14:02:21 UTC
Not that my opinion counts for much, but I'm -1 to consider this a blocker, if it's only triggered by 3rd-party/non-fedora content.

Is it reproducible from content from fedora-only repos?

Comment 8 Kamil Páral 2015-03-25 14:10:17 UTC
Lukas, Richard asked for packagekitd --verbose log, can you please supply it? Thanks.

(In reply to Rex Dieter from comment #7)
> Not that my opinion counts for much, but I'm -1 to consider this a blocker,
> if it's only triggered by 3rd-party/non-fedora content.

I assume this could also be triggered by binary nvidia drivers, they use a similar structure to virtualbox packages. But we haven't tested it yet, it's more difficult to reproduce this with real packages than with fake ones, due to changing repo states.

> 
> Is it reproducible from content from fedora-only repos?

We will try to find out.

Comment 9 Lukas Brabec 2015-03-26 09:02:23 UTC
Created attachment 1006706 [details]
packagekitd --verbose  log

Comment 10 Lukas Brabec 2015-03-26 12:34:00 UTC
I was able to reproduce this with nvidia drivers from rpmfusion.

Version-Release number of selected component (if applicable):
Fedora 21
kernel-3.17.7-300.fc21.x86_64
kernel-3.18.8-201.fc21.x86_64
kernel-3.18.9-200.fc21.x86_64
kernel-core-3.17.7-300.fc21.x86_64
kernel-core-3.18.8-201.fc21.x86_64
kernel-core-3.18.9-200.fc21.x86_64
kernel-modules-3.17.7-300.fc21.x86_64
kernel-modules-3.18.8-201.fc21.x86_64
kernel-modules-3.18.9-200.fc21.x86_64

(downloaded and installed manually to ensure correct version)
kmod-nvidia-3.17.7-300.fc21.x86_64-343.36-1.fc21.1.x86_64
kmod-nvidia-343.36-1.fc21.1.x86_64
xorg-x11-drv-nvidia-343.36-1.fc21.x86_64
xorg-x11-drv-nvidia-libs-343.36-1.fc21.x86_64


yum update output:
================================================================================
 Package        Arch   Version            Repository                       Size
================================================================================
Installing:
 kernel         x86_64 3.19.1-201.fc21    updates                          48 k
 kernel-core    x86_64 3.19.1-201.fc21    updates                          19 M
 kernel-modules x86_64 3.19.1-201.fc21    updates                          17 M
Updating:
 devassistant   noarch 0.9.3-4.fc21       updates                         306 k
 kmod-nvidia    x86_64 1:346.47-2.fc21.1  rpmfusion-nonfree-updates        45 k
 xorg-x11-drv-nvidia
                x86_64 1:346.47-1.fc21    rpmfusion-nonfree-updates       4.0 M
 xorg-x11-drv-nvidia-libs
                x86_64 1:346.47-1.fc21    rpmfusion-nonfree-updates        16 M
Removing:
 kernel         x86_64 3.17.7-300.fc21    @/kernel-3.17.7-300.fc21.x86_64 0.0  
 kernel-core    x86_64 3.17.7-300.fc21    @/kernel-core-3.17.7-300.fc21.x86_64
                                                                           40 M
 kernel-modules x86_64 3.17.7-300.fc21    @/kernel-modules-3.17.7-300.fc21.x86_64
                                                                           17 M
Installing for dependencies:
 kmod-nvidia-3.19.1-201.fc21.x86_64
                x86_64 1:346.47-2.fc21.1  rpmfusion-nonfree-updates       3.7 M
Removing for dependencies:
 kmod-nvidia-3.17.7-300.fc21.x86_64
                x86_64 1:343.36-1.fc21.1  @/kmod-nvidia-3.17.7-300.fc21.x86_64-343.36-1.fc21.1.x86_64
                                                                           15 M


pkcon get-updates output:
Getting updates               [=========================]         
Starting                      [=========================]         
Finished                      [=========================]         
Available   	devassistant-0.9.3-4.fc21.noarch (updates)                  	DevAssistant - Making life easier for developers
Available   	kmod-nvidia-1:346.47-2.fc21.x86_64 (rpmfusion-nonfree-updates)	Metapackage which tracks in nvidia kernel module for newest kernel
Available   	xorg-x11-drv-nvidia-1:346.47-1.fc21.x86_64 (rpmfusion-nonfree-updates)	NVIDIA's proprietary display driver for NVIDIA graphic cards
Available   	xorg-x11-drv-nvidia-libs-1:346.47-1.fc21.x86_64 (rpmfusion-nonfree-updates)	Libraries for xorg-x11-drv-nvidia

Comment 11 Lukas Brabec 2015-03-26 12:34:41 UTC
Created attachment 1006785 [details]
packagekitd --verbose (with nvidia drivers)

Comment 12 Lukas Brabec 2015-03-26 13:56:13 UTC
I didn't find any packages from fedora repo that could cause this problem.

Packages from rpmfusion that will cause this problem:

kmod-VirtualBox-3.17.4-301.fc21.x86_64
kmod-ndiswrapper-3.17.4-301.fc21.x86_64
kmod-nvidia-3.17.4-301.fc21.x86_64
kmod-nvidia-304xx-3.17.4-301.fc21.x86_64
kmod-staging-3.17.4-301.fc21.x86_64
kmod-wl-3.17.4-301.fc21.x86_64
kmod-xtables-addons-3.17.4-301.fc21.x86_64

Comment 13 Kamil Páral 2015-03-26 14:58:18 UTC
(In reply to Lukas Brabec from comment #12)
> Packages from rpmfusion that will cause this problem:
> 
> kmod-VirtualBox-3.17.4-301.fc21.x86_64
> kmod-ndiswrapper-3.17.4-301.fc21.x86_64
> kmod-nvidia-3.17.4-301.fc21.x86_64
> kmod-nvidia-304xx-3.17.4-301.fc21.x86_64
> kmod-staging-3.17.4-301.fc21.x86_64
> kmod-wl-3.17.4-301.fc21.x86_64
> kmod-xtables-addons-3.17.4-301.fc21.x86_64

To be exact, we haven't tested all of them, just virtualbox and nvidia. But these are likely to exhibit the same problem, because they require a particular kernel version and have the same dependency structure (a triangle of kernel <-> metapackage foo <-> package foo-kernel-version).

Comment 14 Adam Williamson 2015-03-30 17:22:37 UTC
Discussed at 2015-03-30 blocker review meeting: http://meetbot.fedoraproject.org/fedora-blocker-review/2015-03-30/f22-blocker-review.2015-03-30-16.04.log.txt .

This is clearly an unfortunate bug, but it's triggered by a dependency structure which cannot exist in the Fedora repos by policy (kernel modules outside of the kernel .src.rpm are not allowed), and seems therefore very strongly a 3rd party repo issue. Fedora has a strong stance of not 'supporting' 3rdparty stuff like this, so we couldn't justify accepting it as a release blocker.

It also appears to be possible to fix it with an update, as the update mechanism itself is not broken.

It's therefore rejected as a blocker, but accepted as a freeze exception issue.

Comment 15 Sergio Basto 2015-04-07 10:58:41 UTC
Hi, unfortunately we can't count on rpmfusion repo because we don't have it for F22
please read [1] for more information 

[1] https://ask.fedoraproject.org/en/question/54642/rpmfusion-on-f22-rawhide/?comment=66707#comment-66707

Comment 16 Kamil Páral 2015-04-07 12:41:48 UTC
*** Bug 1204533 has been marked as a duplicate of this bug. ***

Comment 17 Kamil Páral 2015-04-07 12:43:02 UTC
Please note that in bug 1204533 an independent reporter has confirmed that kmod-wl packages cause the same issue as virtualbox and nvidia packages.

Comment 18 Kalev Lember 2015-06-05 17:12:52 UTC
I debugged this today and https://github.com/hughsie/PackageKit/pull/64 should have the fixes needed to make PackageKit match DNF behaviour with kernel updates.

Comment 19 Sylvain Petreolle 2015-06-05 17:37:25 UTC
(In reply to Kalev Lember from comment #18)
> I debugged this today and https://github.com/hughsie/PackageKit/pull/64
> should have the fixes needed to make PackageKit match DNF behaviour with
> kernel updates.

Awesome news, thanks !

Comment 20 Kamil Páral 2015-06-08 11:24:43 UTC
Created attachment 1036315 [details]
Fedora 22 reproducer version 2

Kalev, thanks a lot. I've found out that the original script broke, because some bodhi updates got deleted and some koji builds as well. I have prepared an updated script for F22 that will hopefully last longer. I actually prepared it last week, but I was waiting for more kernel builds to appear in F22. That hasn't happened, so I just used one from F23 (but it should work the same), and I'm attaching the script now.

It is super-easy to verify this bug with this script. Just install F22 workstation *Live* (has to be Live, because netinst would get you updated packages) into a VM, copy the reproducer script inside and run it as root. It will do everything necessary, then just reboot (necessary, to switch running kernel) and compare dnf vs packagekit. This is the current output:

[root@localhost ~]# dnf clean all && dnf check-update
Cleaning repos: foo
Cleaning up Everything
Foo Repo                                       141 MB/s | 773 kB     00:00    
Last metadata expiration check performed 0:00:00 ago on Mon Jun  8 13:20:02 2015.

kernel.x86_64                        4.1.0-0.rc6.git2.1.fc23                foo
kernel-core.x86_64                   4.1.0-0.rc6.git2.1.fc23                foo
kernel-modules.x86_64                4.1.0-0.rc6.git2.1.fc23                foo
kmod-foo.x86_64                      1-2                                    foo

[root@localhost ~]# pkcon refresh force && pkcon get-updates
Refreshing cache              [=========================]         
Waiting for authentication    [=========================]         
Finished                      [=========================]         
Getting updates               [=========================]         
Finished                      [=========================]         
There are no updates available at this time.


If you update packagekit, you can then test whether the functionality is fixed (or I'll do it, once there's a new F22 build for PK).

Comment 21 Kalev Lember 2015-06-08 14:29:08 UTC
Thanks! I've just kicked off F22 builds with the fixes - testing would be much appreciated.

Comment 22 Fedora Update System 2015-06-08 14:33:39 UTC
PackageKit-1.0.6-6.fc22 has been submitted as an update for Fedora 22.
https://admin.fedoraproject.org/updates/PackageKit-1.0.6-6.fc22

Comment 23 Kamil Páral 2015-06-08 15:28:23 UTC
(In reply to Kalev Lember from comment #21)
> Thanks! I've just kicked off F22 builds with the fixes - testing would be
> much appreciated.

Well, we will need some time to verify this fixed updating from rpmfusion, but it definitely works now with our reproducer script, which is pretty awesome :o)

[root@localhost ~]# pkcon refresh force && pkcon get-updates
Refreshing cache              [=========================]         
Waiting for authentication    [=========================]         
Finished                      [=========================]         
Getting updates               [=========================]         
Finished                      [=========================]         
Available   	kernel-4.1.0-0.rc6.git2.1.fc23.x86_64 (foo)                 	The Linux kernel
Available   	kernel-core-4.1.0-0.rc6.git2.1.fc23.x86_64 (foo)            	The Linux kernel
Available   	kernel-modules-4.1.0-0.rc6.git2.1.fc23.x86_64 (foo)         	kernel modules to match the core kernel
Available   	kmod-foo-1-2.x86_64 (foo)                                   	Dummy summary
Available   	kmod-foo-4.1.0-0.rc6.git2.1-1-2.x86_64 (foo)                	Dummy summary

Comment 24 Fedora Update System 2015-06-08 21:48:28 UTC
gnome-software-3.14.5-2.fc21, PackageKit-1.0.6-1.fc21, libhif-0.2.0-3.fc21, libappstream-glib-0.4.0-1.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/FEDORA-2015-9570/gnome-software-3.14.5-2.fc21,PackageKit-1.0.6-1.fc21,libhif-0.2.0-3.fc21,libappstream-glib-0.4.0-1.fc21

Comment 25 Fedora Update System 2015-06-09 15:17:09 UTC
Package PackageKit-1.0.6-6.fc22:
* should fix your issue,
* was pushed to the Fedora 22 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing PackageKit-1.0.6-6.fc22'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2015-9675/PackageKit-1.0.6-6.fc22
then log in and leave karma (feedback).

Comment 26 Adam Williamson 2015-06-10 22:51:32 UTC
Clearing F22 accepted / nominated freeze exception status as F22 has shipped, per https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Trackers . You may nominate as an Alpha, Beta or Final freeze exception for F23 if desired using the web application - https://qa.fedoraproject.org/blockerbugs/propose_bug (though it is not currently set up for F23) - or by marking the bug as blocking AlphaFreezeException, BetaFreezeException, or FinalFreezeException.

Comment 27 Fedora Update System 2015-06-21 00:06:35 UTC
gnome-software-3.14.5-2.fc21, PackageKit-1.0.6-1.fc21, libhif-0.2.0-3.fc21, libappstream-glib-0.4.0-1.fc21 has been pushed to the Fedora 21 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 28 Fedora Update System 2015-06-21 00:21:40 UTC
PackageKit-1.0.6-6.fc22 has been pushed to the Fedora 22 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.