Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1306992 - PackageKit accumulate over 18GBytes of RPM packages in /var/cache/PackageKit/metadata and fill my root filesystem with unused RPM files
Summary: PackageKit accumulate over 18GBytes of RPM packages in /var/cache/PackageKit/...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: PackageKit
Version: rawhide
Hardware: All
OS: Linux
urgent
high
Target Milestone: ---
Assignee: Richard Hughes
QA Contact: Fedora Extras Quality Assurance
URL: https://bugs.freedesktop.org/show_bug...
Whiteboard: PrioritizedBug
: 1321319 1528757 1573039 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-02-12 11:58 UTC by Yann Droneaud
Modified: 2019-11-18 08:24 UTC (History)
107 users (show)

Fixed In Version: PackageKit-1.1.12-1.fc29
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-11-11 08:14:14 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
bash script to delete redundant rpms (1.62 KB, application/x-shellscript)
2017-07-14 18:57 UTC, D. Hugh Redelmeier
no flags Details
log of run of rpm-cache-prune, the script I attached (1.22 KB, text/plain)
2017-07-14 19:00 UTC, D. Hugh Redelmeier
no flags Details
rpm-cache-prune output (3.77 KB, text/plain)
2017-08-15 12:49 UTC, David Moreau Simard
no flags Details
Script replacing packagekitd in packagekit.service file (1005 bytes, application/x-shellscript)
2017-11-01 09:17 UTC, Standa Laznicka
no flags Details


Links
System ID Private Priority Status Summary Last Updated
FreeDesktop.org 80053 0 'medium' 'RESOLVED' 'PackageKit rpm packages cache folder never cleaned' 2019-11-18 08:22:40 UTC
Red Hat Bugzilla 1321319 0 unspecified CLOSED PackageKit/pkcon does not clean up package cache 2022-05-16 11:32:56 UTC

Description Yann Droneaud 2016-02-12 11:58:54 UTC
My root filesystem is almost full because of .rpm files stored in /var/cache/PackageKit/metadata.

My system is up to date, as I'm installing updates manually through dnf.

PackageKit seems to have the default configuration in /etc/PackageKit/PackageKit.conf as everything is put in comments, especially #KeepCache=false.

I've tried doing "dnf clean all" but it doesn't clean packagekit data and there's no command in pkcon to flush / clean packagekit metadata.

PackageKit should not keep around .rpm files needlessly (eg. older or up to date packages seem worthless) and/or should provide a mean to erase its cache.

Comment 1 Yann Droneaud 2016-02-21 17:18:33 UTC
It's still growing: 16GBytes at this time.
My root filesystem is becoming very short on free space.

Comment 2 Yann Droneaud 2016-02-22 15:43:09 UTC
It's now 17GBytes and my root filesystem is full.

System daemons start complaining:

  auditd[1028]: Audit daemon is low on disk space for logging
  org.freedesktop.Tracker1[2079]: (tracker-store:2619): Tracker-WARNING **: Could not insert FTS text: database or disk is full (strerror of errno (not necessarily related): ....
  auditd[1028]: Audit daemon has no space left on logging partition
  auditd[1028]: Audit daemon is suspending logging due to no space left on logging partition.
  chronyd[1059]: Could not write to temporary driftfile /var/lib/chrony/drift.tmp
  gnome-software-service.desktop[2584]: (org.gnome.Software:2584): Gs-WARNING **: failed to download: cannot download q/qemu-system-x86-2.4.1-7.fc23.x86_64.rpm to /var/cache/PackageKit/metadata/updates/packages/: Curl error (23): Failed writing received data to disk/application for ftp://mir01.syntis.net/fedora/linux/updates/23/x86_64/q/qemu-system-x86-2.4.1-7.fc23.x86_64.rpm [Failed writing body (1116 != 1388)]
  upowerd[1300]: (upowerd:1300): UPower-WARNING **: failed to set data: 

...


At last, PackageKit metadata directory will not grow anymore, but I'm not sure I will be able to safely use my computer ...

Comment 3 Stefano Romano 2016-03-07 15:46:12 UTC
I'm also experimenting same problems.
Looking at folder  /var/cache/PackageKit/metadata I can see many different copies of some rpm packages, in some case 3 different releases as in case of abrt.

Comment 4 Brian J. Murrell 2016-04-19 22:13:57 UTC
Ditto here.  PackageKit seems to be out of control.

Comment 5 Ricardson Williams 2016-05-07 03:18:41 UTC
I have to delete a lot of files in /var/cache/PackageKit/metadata/updates/packages/ almost 2GB....

Comment 6 Volker Sobek 2016-05-07 12:58:26 UTC
I do have the same issue with F24. I suspect that PackageKit does remove rpm packages from /var/cache/PackageKit/24/metadata/updates-testing/packages/ *only after a successful update*.

I can confirm that the last offline-update I just did removed the newest versions of the updated packages from the cache, but older versions of the updated packages have not been removed.

This means if you only use dnf for updates, the cache will grow without any limit.

But even if you use PackageKit the cache can grow without limit: When PackageKit downloads package package-foo.N.rpm of a package and you don't do an offline update before PackageKit downloads package-foo.N+1 (at that point you have two versions of package-foo in the cache), the next time you do an offline update only package-foo.N+1.rpm would be removed from the cache, and package-foo.N would remain in the cache.

One solution could be that PackageKit removes old RPM packages from the cache when downloading a newer version.

Comment 7 Joshua Roys 2016-05-10 14:04:35 UTC
This is very noticeable on VMs with small root partitions.

[root@localhost packages]# ls -l1tr libreoffice-core*
-rw-r--r--. 1 root root 77234618 Nov 13 10:31 libreoffice-core-5.0.3.2-4.fc23.x86_64.rpm
-rw-r--r--. 1 root root 77239518 Dec  2 06:24 libreoffice-core-5.0.3.2-7.fc23.x86_64.rpm
-rw-r--r--. 1 root root 77239878 Dec  5 06:24 libreoffice-core-5.0.3.2-11.fc23.x86_64.rpm
-rw-r--r--. 1 root root 77237762 Dec 15 06:24 libreoffice-core-5.0.3.2-12.fc23.x86_64.rpm
-rw-r--r--. 1 root root 77268554 Dec 25 06:25 libreoffice-core-5.0.4.2-3.fc23.x86_64.rpm
-rw-r--r--. 1 root root 77279386 Feb 17 06:35 libreoffice-core-5.0.5.2-1.fc23.x86_64.rpm
-rw-r--r--. 1 root root 77286578 Feb 25 06:38 libreoffice-core-5.0.5.2-2.fc23.x86_64.rpm
-rw-r--r--. 1 root root 77304962 Mar 12 06:08 libreoffice-core-5.0.5.2-4.fc23.x86_64.rpm
-rw-r--r--. 1 root root 77315190 Mar 15 06:12 libreoffice-core-5.0.5.2-6.fc23.x86_64.rpm
-rw-r--r--. 1 root root 77311350 Apr 21 06:25 libreoffice-core-5.0.6.1-1.fc23.x86_64.rpm
-rw-r--r--. 1 root root 77318598 May  7 06:25 libreoffice-core-5.0.6.2-3.fc23.x86_64.rpm
[root@localhost packages]# du -sh $PWD
4.7G	/var/cache/PackageKit/metadata/updates/packages

Comment 8 Joshua Wilson 2016-05-16 19:02:10 UTC
I'm on F23 and I'm seeing that same thing.

I do all my updates with dnf on the cli. 

1) What does PackageKit do in F23? 
2) Can I remove PackageKit?
3) Can I safely remove the contents of /var/cache/PackageKit/metadata/updates/packages ?

Comment 9 Joshua Wilson 2016-05-17 03:43:02 UTC
Here is what I found out and how I'm working around the issue. PackageKit is used by GNOME. If you use Yum or DNF on the CLI then don't need these. You can remove the .rpm files in /var/cache/PackageKit/metadata/updates/packages and set PackageKit to not store them any longer.

There is a setting in the file /etc/PackageKit/PackageKit.conf

    # Keep the packages after they have been downloaded
    #KeepCache=false

As root, remove the hash mark on this configuration option and the packages will not be saved.

Comment 10 Adam Pribyl 2016-06-12 20:43:47 UTC
 pkcon refresh force -c -1
is your friend.

Comment 11 mwp.junk 2016-10-22 17:05:39 UTC
pkcon refresh force -c -1 is a temporary solution that I don't like.

For me, I either:

- set /etc/PackageKit/PackageKit.conf

    # Keep the packages after they have been downloaded
    KeepCache=false

- sudo dnf remove PackageKit* gnome-software

The latter being my favorite...

Pretty poor design, having two update mechanism that have their own metadata. Even worse design enabling both, and just absolutely insane not asking the user during install which they prefer.

Comment 12 Fedora End Of Life 2016-11-24 15:31:38 UTC
This message is a reminder that Fedora 23 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 23. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '23'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 23 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 13 Fabio Olive Leite 2016-12-02 12:59:53 UTC
This still happens on Fedora 24, bumping the bz version.

Comment 14 Sandro Bonazzola 2016-12-13 20:10:51 UTC
Still an issue on fc25 as well

Comment 15 Clifford Perry 2016-12-29 13:03:50 UTC
Adding myself to cc, since while doing a F23 -> F25 upgrade I had to first delete over 22G of content out of the PackageKit cache (so that system had enough disk space in / to complete upgrade). 

I don't know (yet) if PackageKit in F25 behaves just as bad. 

Cliff

Comment 16 Thomas Mueller 2017-01-02 07:44:30 UTC
My F25 system has KeepCache=false in PackageKit.conf and /var/cache/PackageKit is now about 5G in size.

Comment 17 Thomas Mueller 2017-01-02 08:02:07 UTC
    sudo pkcon refresh force -c -1

did not free space in /var/cache/PackageKit. not only newest rpm where stored but also intermediate versions. as i never install update by PackageKit (but by dnf update) the KeepCache=false in PackageKit.conf seems not to be effective.

I've used now:

    sudo find /var/cache/PackageKit/ -name "*.rpm" -delete

seems like setting download-updates [1, 2] to false is a good idea:

    gsettings set org.gnome.software download-updates false

hoping it'll help stopping auto-download of packages.

Shouldn't this auto-download be opt-in instead of opt-out by default?


[1] https://bugzilla.gnome.org/show_bug.cgi?id=768632

[2] Howto: https://www.lguruprasad.in/blog/2015/05/13/disabling-automatic-download-of-software-updates-in-gnome-3-14-on-debian-jessie/

Comment 18 Jean-François Fortin Tam 2017-02-14 17:00:45 UTC
Folks, this is essentially https://bugs.freedesktop.org/show_bug.cgi?id=80053 (which I filed ages ago). Would love to see someone working on a patch for that.

Comment 19 D. Hugh Redelmeier 2017-02-19 18:59:10 UTC
It seems especially dumb to keep outdated RPMs in the cache.  It seems very improbable that they would be useful to anyone.

Here's a posting I made almost a year ago.  It includes a hacky almost-script to delete the obsolete packages.  Surely this should be automatic (possibly an option).

<https://lists.fedoraproject.org/pipermail/users/2016-March/469483.html>

Comment 20 Christian Stadelmann 2017-03-24 16:18:03 UTC
Please nominate this bug as PrioritizedBug following the steps in https://fedoraproject.org/wiki/Fedora_Program_Management/Prioritized_bugs_and_issues_-_the_process

Comment 21 Fabio Olive Leite 2017-03-24 18:53:12 UTC
This is in a recent Fedora 25 installation.

$ cat /etc/fedora-release 
Fedora release 25 (Twenty Five)

$ sudo du -sh /var/cache/PackageKit/25/
980M	/var/cache/PackageKit/25/

$ ls /var/cache/PackageKit/25/metadata/updates/packages/libreoffice-core*
.../metadata/updates/packages/libreoffice-core-5.2.6.2-2.fc25.x86_64.rpm
.../metadata/updates/packages/libreoffice-core-5.2.6.2-3.fc25.x86_64.rpm
.../metadata/updates/packages/libreoffice-core-5.2.6.2-4.fc25.x86_64.rpm

Clearly it is still maintaining multiple versions of older packages in the cache.

Comment 22 Jan Kurik 2017-04-19 14:43:19 UTC
We have postponed approval of this bug for the Prioritized list as we have not get a quorum for the approval. If not solved, we will get back to this bug on the next evaluation meeting.

Comment 23 Brian J. Murrell 2017-05-20 14:50:17 UTC
My 4461528 /var/cache/PackageKit/25/metadata/updates/packages is 4461528 KB (4.4GB!) big with lots of old, stale packages in it.

Can we please get this fixed?  It's becoming a disk space management nightmare.

Comment 24 Brian J. Murrell 2017-05-20 21:19:25 UTC
I've just reduced /var/cache/PackageKit/25/metadata/updates/packages to 699404 (almost a tenth of what it was) by removing all of the old cruft in it.

Comment 25 Jan Kurik 2017-05-25 08:32:16 UTC
This bug has been accepted as a Prioritized bug: https://fedoraproject.org/wiki/Fedora_Program_Management/Prioritized_bugs_and_issues_-_the_process

Comment 26 Jan Kurik 2017-05-25 16:17:50 UTC
Dan, we discussed this bug on "Prioritized bugs evaluation meeting" https://meetbot.fedoraproject.org/teams/fedora_prioritized_bugs_and_issues/fedora_prioritized_bugs_and_issues.2017-05-24-14.00.log.html#l-23
and we come to conclusion that we need to synchronize the work on this bug with work on libdnf component. May I ask you please to check this bug and propose some steps how we can move on ?

Comment 27 Yann Droneaud 2017-05-29 16:29:38 UTC
Bug #1321319 is a duplicate of this one.

Comment 28 Máirín Duffy 2017-06-27 13:36:47 UTC
fwiw this happened to me, 25GB of packagekit cache until my system stopped working right then i figured it out thanks to baobab. f25 system that had been upgraded for the past year or two from f22 or f23. cache had f23 pkgs in it.

Comment 29 Jean-François Fortin Tam 2017-06-27 14:48:22 UTC
*** Bug 1321319 has been marked as a duplicate of this bug. ***

Comment 30 D. Hugh Redelmeier 2017-07-14 18:57:17 UTC
Created attachment 1298585 [details]
bash script to delete redundant rpms

This is what I do until this bug gets fixed.  It is a hack.

Comment 31 D. Hugh Redelmeier 2017-07-14 19:00:38 UTC
Created attachment 1298587 [details]
log of run of rpm-cache-prune, the script I attached

This run is on a system after upgraded to Fedora 26 using dnf, plus a couple of dnf updates.

Notice how much cruft was removed.

This should not be necessary.

Comment 32 francko 2017-08-02 12:00:16 UTC
still an issue on a fresh installed (not updated) Fedora 26 system

as a temporary solution i removed the complete folder /var/cache/PackageKit folder

with dconf-editor i turned off the automatic download of packages

hope it helps

Comment 33 Sam Tuke 2017-08-10 13:06:25 UTC
Hit by this issue again today. I followed the advice of mwp.junk and removed packagekit (and KDE Desktop that is a dependency). Sad solution to what seems like a simple problem, but over the years having the root partition fill up and break systems because of this is an unacceptable alternative.

Thanks D. Hugh Redelmeier for your script; tried it first but got:

./rpm-cache-prune: line 29: cd: /var/cache/PackageKit/2[0-9]/metadata/*/packages/: No such file or directory

Comment 34 cornel panceac 2017-08-12 05:43:07 UTC
$ sudo pkcon refresh force -c -1
...
Fatal error: Error when getting information for file “/var/cache/PackageKit/26/metadata/rpmsoftwaremanagement-dnf-nightly/repodata/f70e09a1e6b14110211fff739038a5b90e31619e8d900ac8c8b079cf0ef6d9c8-appstream.xml.gz”: No such file or directory

Comment 35 cornel panceac 2017-08-12 05:50:14 UTC
This happens while the corresponding .repo file had 'enabled=0' in it.
I have removed the .repo file and it worked.

Comment 36 D. Hugh Redelmeier 2017-08-12 16:14:38 UTC
Sam Tuke: Thanks for trying the script.  This error means that there is no directory matching this pattern.  Perhaps it is a permission problem (but that doesn't happen on my systems).  Or there just are no such directories on your system.  If so, where are these RPMs being cached on your system?

Comment 37 David Moreau Simard 2017-08-15 12:49:22 UTC
Created attachment 1313662 [details]
rpm-cache-prune output

I'm also running into this issue and found this bug while troubleshooting why PackageKit cache made my root partition fill up to 99%.

See the attached output from the rpm-cache-prune script (thanks for that).
It seems like, at the very least, cache from the previous releases of Fedora should be deleted during the upgrade process. The output makes obvious that I was carrying a bunch of cache from F24 and F25 -- I'm currently running F26.

Comment 38 Lars S. Jensen 2017-09-03 10:58:03 UTC
This bug is not good for the lift-time of small SSD/SD device if the device get fill with useless temporary data because the SSD's build-in wear-leveling will caused an extensive erase-compact cycles. 

It is related to GNOME Desktop and on XFCE desktop it's not a issue except if GNOME Software is in Application Autostart session. 

This bug should have high priority and severity and all version of F2x Workstation.

As a minimum should the default setup prevent filling up the filesystem.

Comment 39 Jan Kurik 2017-09-27 14:35:22 UTC
Richard, is there any progress on this bug ? We would be happy if this can be fixed for F27 Final release.

Comment 40 rockonthemoonfm 2017-10-03 08:02:24 UTC
Agreed this has to de fixed. 

Bought a new SSD two months ago, gave root partition 10Gb, couldn't update anymore because of the following folders taking up an out-of-control disk space:

1 /var/cache/PackageKit/26/metadata/fedora/packages 
2 /var/log/journal
3 /var/lib/systemd/coredump
4 /var/lib/flatpak/repo/objects

Definitely not a selling point here.
The rest is good (apart dnf/packagekit redundancy;)

Comment 41 Tuomas Kuosmanen 2017-10-16 06:44:40 UTC
Just a bump that this bit me also. Had upgraded this laptop through a few releases and thus started to get warnings of <1GB free on root filesystem.

This was "just partition my disk for me" installation with 50GB root filesystem.

I do most of my updates with dnf on commandline, so maybe the packagekit downloaded RPMs are not being used and/or cleaned up?

Comment 42 Standa Laznicka 2017-11-01 09:17:12 UTC
Created attachment 1346387 [details]
Script replacing packagekitd in packagekit.service file

I tried many ways mentioned in this BZ to stop packagekit service from running but packagekit still runs after system start.

I am attaching the "pkgkitfixup.sh" script which will tell systemd to run a "dumb" script accepting and discarding any options it will get, reminding you that it's a mere workaround for this BZ in the system journal.

Unfortunately any upgrade to the PackageKit package will replace the workaround in the packagekit.service file. On the other hand, you can easily get rid of this workaround by running `dnf reinstall PackageKit`.

To remove your redundant rpms, you can use the scripts already attached to this BZ.

Comment 43 Antoine Cotten 2017-11-01 09:35:32 UTC
There is no need for workarounds to disable PackageKit. Just mask the unit with

systemctl mask packagekit.service

Changes will be persisted inside /etc/systemd, which package updates do not affect.

Comment 44 Fedora End Of Life 2017-11-16 18:49:47 UTC
This message is a reminder that Fedora 25 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 25. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '25'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 25 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

Comment 45 Karl Hastings 2017-11-16 18:53:33 UTC
Bumping 'Version' to 26 since this is still an issue there. (I don't know if it's still an issue in f27, haven't looked into it.)  We don't want the F25 EOL to close this BZ.

Comment 46 David Binderman 2017-11-16 19:05:29 UTC
>I don't know if it's still an issue in f27, haven't looked into it

I just upgraded to F27 and it looks to me to still be an issue.

>Pretty poor design, having two update mechanism that have their own metadata. >Even worse design enabling both, and just absolutely insane not asking the user >during install which they prefer.

+1

It is a fundamental feature of all Unix systems that you don't pay
for what you don't use. I don't use PackageKit, but it still persists
in downloading Megabytes of data that I don't need or want.

Removing PackageKit seems a big improvement to me. 

It would be a valid enhancement to have PackageKit's cacheing policy under 
easier user control, like being able to easily switch it off, or control
what %age of the disk it uses.

Comment 47 John Thacker 2017-11-18 15:05:27 UTC
For me at least on F26,

# pkcon refresh force

will delete from the cache packages that have already been installed (whether through PackageKit or dnf/yum) but only from currently enabled repositories. What it won't do is delete from old repositories, such as the 24 or 25 directories after one has updated to 26.

It really should be run automatically periodically by default, rather than holding onto an enormous cache of already installed packages.

Comment 48 Fabio Olive Leite 2017-11-23 13:40:08 UTC
Yep, I can confirm that this workaround also mitigates the storage issue (not the wasted bandwidth one, though) in a Fedora 25 laptop.

$ sudo dnf update
...
$ sudo pkcon refresh force

After this, "sudo find /var/cache/PackageKit/ -name \*.rpm" does not find any RPM files in the cache.

The metadata PackageKit is storing for Fedora 25, 26 and 27 currently take up 466M, which is not great, but at least not the multi-gigabyte reported before.

Comment 49 John Dennis 2017-12-14 19:59:53 UTC
Ouch! This just bit me too. I tried to use PackageKit to upgrade to F27 and discovered after a few minutes my root filesystem was full. After a bit of investigation I discovered half of the filesystem was being consumed by /var/cache/PackageKit! There were files and directories going back to F23.

Since I had no idea of what inconsistent state the cache was in after the filesystem filled up I removed everything under /var/cache/PackageKit, rebooted, / was back to 50% full and after starting the PackageKit GUI it immediately reinitialized by downloading the repo metadata and repopulating  /var/cache/PackageKit (with just f26 and f27), so at least that part is good.

PackageKit needs to clean up after itself and not leave a lot of cruft around.

As a developer I often use dnf on the command line. Somewhere someone mentioned this will cause 2 independent package caches to be maintained. Not sure if that's the case or not but if so it obviously not good.

Comment 50 JOhn Westerdale 2017-12-27 21:35:56 UTC
Yum clean all doesn't clean all. Have to delete many dirs and start over again.
/var/log/yum is still updating, even though its deprecated by dnf?
what is /var/cache/hawkey doing, and how does that get filled and cleaned?
if "dnf system-upgrade download --refresh --releasever=27" fails, what do i need to clean out? 

- Can we make a clean break with Yum if its no longer supported - and be done with it? 
- Can we identify a way to clean out /var/cache/PackageKit? perhaps a dedup?
- Lets tighten up this ship.

Comment 51 Joseph D. Wagner 2017-12-30 01:43:18 UTC
+1 to #50

Comment 52 Miroslav Suchý 2018-01-21 11:18:50 UTC
Relevant: https://github.com/xsuchy/fedora-upgrade/pull/20
But fedora-upgrade is not an official tool. So this issue is still valid.

Comment 53 Chris Murphy 2018-01-22 04:50:35 UTC
(yay btrfs snapshots for making this a lot easier to troubleshoot)

I'm on Fedora 27 Workstation and this problem still exists out of the box. But I've found that uncommenting #KeepCache=false from /etc/PackageKit/PackageKit.conf does fix this problem, although it may not remove already applied RPMs.

But what I can confirm is that after PK/GNOME Software downloads and applies 1G of RPMs to an old snapshot I had from November, all of those applied RPMs have been deleted prior to the reboot from the offline update; i.e. they were applied and deleted during the offline update process.

I'm really confused why the KeepCache option is set to true by default.

Comment 54 Luya Tshimbalanga 2018-01-22 05:37:36 UTC
(In reply to Chris Murphy from comment #53)
> (yay btrfs snapshots for making this a lot easier to troubleshoot)
> 
> I'm on Fedora 27 Workstation and this problem still exists out of the box.
> But I've found that uncommenting #KeepCache=false from
> /etc/PackageKit/PackageKit.conf does fix this problem, although it may not
> remove already applied RPMs.
> 
> But what I can confirm is that after PK/GNOME Software downloads and applies
> 1G of RPMs to an old snapshot I had from November, all of those applied RPMs
> have been deleted prior to the reboot from the offline update; i.e. they
> were applied and deleted during the offline update process.
> 
> I'm really confused why the KeepCache option is set to true by default.

I think KeepCache=true is set for failsafe in case the offline update somehow broke. There should have an option for expiring time to keep these package by default.

Comment 55 Richard Hughes 2018-01-24 12:51:08 UTC
(In reply to Chris Murphy from comment #53)
> I'm on Fedora 27 Workstation and this problem still exists out of the box.
> But I've found that uncommenting #KeepCache=false from
> /etc/PackageKit/PackageKit.conf does fix this problem, although it may not
> remove already applied RPMs.

I'm confused -- if the key is commented, it should be read as FALSE. Uncommenting it *should* have done nothing.

> I'm really confused why the KeepCache option is set to true by default.

It should be the opposite... Investigating.

Comment 56 Richard Hughes 2018-01-24 13:47:16 UTC
I've done some local testing for this bug. Both installing and removing remove the correct file with the default (commented out) KeepCache=false as expected. You can verify this by breaking on g_file_delete() inside PackageKit. I've also verified that dnf_context_get_keep_cache() returns the right value (FALSE) when run in a DnfTransation. I don't see how uncommenting KeepCache changes anything at all. Would a simple "delete anything older than 1 month in /var/cache/PackageKit" solve this bug?

Comment 57 Gustavo Maciel Dias Vieira 2018-01-24 13:55:23 UTC
KeepCache should not be relevant. In my observations the problem is caused when a package is downloaded by PackageKit and 1) it is installed by DNF in the command line before an upgrade in Gnome Software, or 2) a new version is downloaded by PackageKit before an upgrade in Gnome Software.

So, yes, purging the cache from time to time would solve it.

Comment 58 jeremy9856 2018-01-24 14:05:11 UTC
(In reply to Richard Hughes from comment #56)
> Would a simple "delete anything older than 1 month in /var/cache/PackageKit"
> solve this bug?

What about deleting the cached packages that are not relevant anymore (newer version cached, update done with dnf, etc... ) by checking updates from time to time ?

Comment 59 Fabio Olive Leite 2018-01-24 15:55:29 UTC
(In reply to jeremy9856 from comment #58)
> What about deleting the cached packages that are not relevant anymore (newer
> version cached, update done with dnf, etc... ) by checking updates from time
> to time ?

Yeah, I would expect a "relevance test" to be the desired outcome here.

It is fine to keep some downloaded packages, as long as they are still relevant updates to the currently installed package base.  Periodic checks should exclude older versions of the same package, cached packages that are already updated (via DNF or elsewhere), and finally cached updates to packages that have since been removed.

Comment 60 Michael Catanzaro 2018-01-24 18:14:54 UTC
In particular: packages should not be cached forever if the user decides to always apply updates with dnf rather than PackageKit.

Comment 61 John Dennis 2018-01-24 18:26:00 UTC
Re comment #57, I also mix command line use of DNF with the package updater GUI. Are these two tools unaware of each other's actions? If so that would be very bad.

Comment 62 Adam Pribyl 2018-01-25 18:06:53 UTC
Packagekit updates the dnf transaction db after update/installation, however there is no history, no log or whatever from packagekit actions. You can not revert the package installed by PK... It is not using dnf as a backend, its actions are independen on dnf. I removed it everywhere.

Comment 63 Joseph D. Wagner 2018-01-25 22:46:04 UTC
+1 to comments #57 - #60, and the insights offered by comments #61 - #62

Comment 64 David Hill 2018-01-28 14:42:30 UTC
I was dealing with this issue for the past months/years and it today I got annoyed by it and commenting here.  I'm also not using the PackageKit service as I do "yum update --exclude dom4j" every now and then  ... Is there a reason why there's not an automatic cleanup process in PK that would cleanup things automagically when disk usage is above a given threshold ?

Comment 65 David Hill 2018-01-28 14:44:31 UTC
Or even a dnf/yum hook script that would purge PK cache if needed ?  I still like having the older release packages because when one update breaks something, rollbacking to the previous version is faster/easier ... but having 30GB of RPMs on a small SSD where /home is encrypted is a bit overkill  IMHO.

Comment 66 Chris Murphy 2018-01-28 19:11:59 UTC
Contrary to c53, now I see PK is downloading RPMs throughout the week, but they're not being installed nor am I notified of them. By the time the notification happens, some downloaded RPMs are superceded by new versions which have been downloaded. The new versions are installed and cleanedup, but the ones downloaded and ignored are not cleaned up.

In my view, the lack of cleanup of these stale RPMs is less bad than the fact PK wasted bandwidth needlessly downloading RPMs it had no intention of installing.

Many ISPs in the U.S. have vague caps on max downloads per month, some of them silently claim you're over the cap but do nothing until you go over it by some undefined amount, and then you get nailed. And crap bandwidth is also widespread, so again having something downloading shit unnecessarily is just a huge waste of a limited resource.

Comment 67 Carwyn Edwards 2018-01-28 19:22:47 UTC
Another use case worth mentioning, myself being an example. I use dnf exclusively for updates and most installations however Gnome Software is very useful for browsing and the occasional installation. All three of my Fedora machines ended up with 20GB+ of unused, uncleared packages. I had no idea they were there until a chance df output alerted me to some unusual behaviour. This package cache represented over 50% of the installed disk footprint on two of the machines.

In my case I'm tempted to remove PackageKit but then I also lose useful functionality of Gnome Software.

Comment 68 Chris Murphy 2018-01-28 19:38:19 UTC
(In reply to Carwyn Edwards from comment #67)

The CLI solution for this problem is 'gsettings set org.gnome.software download-updates false'

Since a long time ago there should have been a visible toggle for this in GNOME Software, and/or an opt-in/out in GNOME Initial Setup.

Comment 69 Richard Hughes 2018-01-28 22:02:32 UTC
The design team wanted to only show updates in the gnome-software update panel that had been downloaded, depsolved, and were ready to be deployed offline. They also wanted to only notify the user about pending updates infrequently (perhaps only 1/week), unless there was a pending security update. Fedora is somewhat of a package firehose, with until recently no real batching of non-urgent updates. PackageKit and DNF do not share a package cache (and now dnf is being rewritten in C++, that's looking further and further away...) so you don't use (and thus delete) the PK-downloaded caches when using dnf on the CLI.

At lease one of those constraints has to be relaxed, to avoid the "download package and not use it unless manually going to the updates panel".

It probably sounds like I'm making excuses, but I'm really trying my best to do my best with the guidance I'm being given. The one bug that is here is that libdnf should delete the old version of a package when downloading the new version, although that's not really possible to work out reliably due to package renames, splits and merges making a simple "check if older" into a heuristic, not a rule. When everyone is back from Devconf we can sit down and work out a plan.

Comment 70 mwp.junk 2018-01-28 23:50:11 UTC
(In reply to Richard Hughes from comment #69)
> The design team wanted to only show updates in the gnome-software update
> panel that had been downloaded, depsolved, and were ready to be deployed
> offline. They also wanted to only notify the user about pending updates
> infrequently (perhaps only 1/week), unless there was a pending security
> update. Fedora is somewhat of a package firehose, with until recently no
> real batching of non-urgent updates. PackageKit and DNF do not share a
> package cache (and now dnf is being rewritten in C++, that's looking further
> and further away...) so you don't use (and thus delete) the PK-downloaded
> caches when using dnf on the CLI.
> 
> At lease one of those constraints has to be relaxed, to avoid the "download
> package and not use it unless manually going to the updates panel".
> 
> It probably sounds like I'm making excuses, but I'm really trying my best to
> do my best with the guidance I'm being given. The one bug that is here is
> that libdnf should delete the old version of a package when downloading the
> new version, although that's not really possible to work out reliably due to
> package renames, splits and merges making a simple "check if older" into a
> heuristic, not a rule. When everyone is back from Devconf we can sit down
> and work out a plan.

With all due respect, Mr. Hughes, it doesn't matter what the reasons are as to why things are the way they are. This is a **HUGE** issue and it needs to be solved. There is no excuse or dancing around it to make things better.

It was a major design flaw to think that PackageKit and DNF could live side-by-side and not talk without repercussions. If packagekit doesn't share a package cache with dnf, it needs to be disabled until it does. End of story.

Comment 71 mwp.junk 2018-01-29 00:14:44 UTC
Its been almost 2 years and the people responsible are still sitting around doing nothing. I guess my original actions of simply removing all traces of PackageKit and Gnome Software are the best I can hope for; I am done with this bug.

Comment 72 David Binderman 2018-01-29 08:53:48 UTC
(In reply to mwp.junk from comment #71)
> Its been almost 2 years and the people responsible are still sitting around
> doing nothing. 

Indeed. 88 of us are watching this bug waiting for some useful action.

In my view, this speaks volumes about Redhat's ability to respond to and fix bugs. Submitting further bug reports looks largely pointless.

I don't think I'll be buying anything from Redhat anytime soon.

Comment 73 jeremy9856 2018-01-29 09:17:22 UTC
(In reply to Richard Hughes from comment #69)
> When everyone is back from Devconf we can sit down
> and work out a plan.

Thanks Richard for taking care of this. I don't know if other distro are impacted but if it's the case I hope that the fix will work for them too.

https://bugs.freedesktop.org/show_bug.cgi?id=80053

(In reply to mwp.junk from comment #71)
> Its been almost 2 years and the people responsible are still sitting around
> doing nothing. 

Its been almost 4 years :D
https://bugs.freedesktop.org/show_bug.cgi?id=80053

Comment 74 jeremy9856 2018-01-29 09:34:57 UTC
(In reply to Carwyn Edwards from comment #67)
> Another use case worth mentioning, myself being an example. I use dnf
> exclusively for updates and most installations however Gnome Software is
> very useful for browsing and the occasional installation. All three of my
> Fedora machines ended up with 20GB+ of unused, uncleared packages. I had no
> idea they were there until a chance df output alerted me to some unusual
> behaviour. This package cache represented over 50% of the installed disk
> footprint on two of the machines.

Of course you didn't see it, they remove the free space available for disks in Nautilus. Its been almost 6 years we ask to reintroduce it.

https://bugzilla.gnome.org/show_bug.cgi?id=684943

Comment 75 Richard Hughes 2018-01-29 12:41:12 UTC
(In reply to David Binderman from comment #72)
> In my view, this speaks volumes about Redhat's ability to respond to and fix
> bugs.

First, it's Red Hat, nor Redhat. Second, PackageKit is an upstream project that anyone can contribute towards. It's not a RH project at all. I also don't think it's a huge secret that PackageKit isn't the new hotness, and most of us are focusing on an ostree atomic core with flatpak applications. The desktop of the future probably isn't going to include PackageKit, so doesn't get the same kind of maintainer time as it did.

Most of the time I spend on PackageKit these days is in the evenings and weekends, so thank you all for making me feel so useless.

Comment 76 Matthias Clasen 2018-01-29 14:37:13 UTC
> Most of the time I spend on PackageKit these days is in the evenings and
> weekends, so thank you all for making me feel so useless.

Ignore the heckling. i value the work you do.

> At lease one of those constraints has to be relaxed, to avoid the "download 
> package and not use it unless manually going to the updates panel"

I don't think that is necessarily true. I've discussed a bit with Kalev how we can handle the cache better, even if we can't avoid caching updates. Suggestions:

1) Whenever we update the cache, check if the installed version of a package is newer than the cached one. if so, drop the package from the cache / don't download it

2) When downloading updates, remove all older versions of the same package from the cache

Taken together, this should ensure that we don't grow the cache without bounds (we'll approach a limit size of 1 newer version of each installed package) and that we don't download outdated packages

Comment 77 Rex Dieter 2018-01-29 14:52:48 UTC
Before negativity here runs unchecked, a friendly reminder on expected behavior:
https://getfedora.org/code-of-conduct.html.en

and please stay on topic: anything beyond the technical aspects of this issue is likely not well-suited for this forum (bugzilla), and will only lower the signal-to-noise ratio.

Comment 78 Joseph D. Wagner 2018-02-16 18:37:01 UTC
Due to the differences in design goals outlined in comment #69, I decided to uninstall PackageKit until dnf and PackageKit either 1) used a shared cache, or 2) PackageKit removes already updated rpms.

IMHO, PackageKit is a waste of my bandwidth and storage space if it can't work with dnf.

Comment 79 John Dennis 2018-02-16 19:22:55 UTC
Given the fundamental design issues (incompatible with the system installer, e.g. dnf) and the problems it's causing shouldn't this package be removed from the distribution until the design issues are resolved?

Comment 80 Jean-François Fortin Tam 2018-02-16 20:40:18 UTC
> Given the fundamental design issues (incompatible with the system installer, e.g. dnf)
> and the problems it's causing shouldn't this package be removed from the distribution
> until the design issues are resolved?

My rough understanding (just guessing), from my experience removing PackageKit, is that GNOME Software depends on it directly. If you remove PackageKit, you end up with no package search/install/remove GUI for GNOME-based workstation users.

The odd thing with Fedora is that, from what I've seen over the past ten years or so as a workstation user, its weak points have always been the installer and the package management GUIs. I've always been surprised how they're not the #1 priority as, in my personal viewpoint, package management is kind of the whole point and differentiator of desktop Linux distros ;)

Comment 81 Michael Catanzaro 2018-02-16 21:00:34 UTC
(In reply to John Dennis from comment #79)
> Given the fundamental design issues (incompatible with the system installer,
> e.g. dnf) and the problems it's causing shouldn't this package be removed
> from the distribution until the design issues are resolved?

Yeah, it's required for GNOME Software. We could flip the argument around and get rid of dnf instead on the basis that it's incompatible with PackageKit, but that wouldn't be very helpful I don't think.

At this point, useful contributions to this bug would entail help to implement the following:

(In reply to Matthias Clasen from comment #76)
> 1) Whenever we update the cache, check if the installed version of a package
> is newer than the cached one. if so, drop the package from the cache / don't
> download it
> 
> 2) When downloading updates, remove all older versions of the same package
> from the cache

Comment 82 Chris Murphy 2018-02-16 22:39:27 UTC
The path of least resistance is if PackageKit can clean up periodically so at least users aren't running out of space due to stale packages accumulating.

What we should have done, hindsight being 20/20, is when the yum/dnf folks decided they weren't going to rename dnf to yum as promised, is change the documentation and retrain users for Workstation to use pkcon at the command line. But this ship has sailed.

Comment 83 Luya Tshimbalanga 2018-02-17 06:11:29 UTC
(In reply to Matthias Clasen from comment #76)
> 1) Whenever we update the cache, check if the installed version of a package
> is newer than the cached one. if so, drop the package from the cache / don't
> download it
> 
> 2) When downloading updates, remove all older versions of the same package
> from the cache

Which bring a question, what is the current status of Unified database for DNF https://fedoraproject.org/wiki/Changes/Unified_database_for_DNF ?

Comment 84 Kalev Lember 2018-03-05 19:12:17 UTC
OK folks, some good news! PackageKit 1.1.9 now has cleanup for old cache directories, deleting e.g. /var/cache/PackageKit/{23,24,25,26,27}/ when running on Fedora 28:

https://github.com/hughsie/PackageKit/commit/4aab5fd604e7638cb29f329c2971606e22c167f8

There's more work to do in this area, but this should be a good first step. PackageKit 1.1.9 is in F28 and soon coming to F27 as well.

Comment 85 Flo H. 2018-03-07 16:09:18 UTC
Kalev, question on #84:

When running Fedora 28, one can expect to have packages from other releases, likely 26, 27, and maybe even 29, no?

My F26 system has content in all of /var/cache/PackageKit/{25,26,27}

So, is it the right approach to just delete data from older releases? If there hasn't happened a mass rebuild, Fedora releases usually carry old packages. Correct me if I am wrong (I am just an ordinary end-user, no IT guy, nor developer or something similar).

Comment 86 Kalev Lember 2018-03-07 17:08:18 UTC
Hi Flo! Yes, should be fine to delete all of them. We put .fc25, .fc26, .fc27 etc packages that belong to Fedora 28 to /var/cache/PackageKit/28 and don't load anything from the older directories.

Comment 87 Hirm Loh 2018-03-09 10:12:48 UTC
hey, i have same problem also and i am not IT guy, so what should i do if my /var is full now ?? what commen number that solve the problem ??
what command to delete /var/cache/ ??
thx

Comment 88 JanS 2018-03-09 12:03:42 UTC
This bug is very baaad.
Not only normal users are signing up to this bug report (which is uncommon, unless the bug is common _and_ annoying), but also:

I am getting random questions by coworkers in my company about having no disk space on their laptops. And they, understandably, do not know why?
Although one coworker did figured out which was the offending directory, she tried to right click it and "Move to Trash", which obviously didn't work! 

They are normal users who just want to get the job done. But somehow there is a secret agent on their computer that just eats all the space regardless of anything. And I do not know what to tell them - not only why, but also what to do about it?

I am personally not using PackageKit at all, only dnf. But what should I advise to coworkers?

Earlier-mentioned mitigation for setting KeepCache=false in /etc/PackageKit/PackageKit.conf did not work for me, so I do not know if it will be effective for them, hence, I am unsure if I should recommend this as a solution.

I am also afraid to advise this:
  gsettings set org.gnome.software download-updates false
as I do not know if PackageKit will keep remind people to update their laptops if this is set..?

This command:
    sudo find /var/cache/PackageKit/ -name "*.rpm" -delete
is only a temporary relief for the disk space, but does not change the PackageKit behaivior.

I would not advise for un-installing PackageKit on their laptops, since these are normal users, so they will not remember to update by themselves.

So in summary:
IF this bug cannot be fixed! (two years and counting...), THEN please provide a clear guide for non-technical users..
(should appear in the Release Notes for Fedora 28 as a known bug, eg. in Common_F28_bugs document).

Comment 89 D. Hugh Redelmeier 2018-03-09 14:41:34 UTC
Comment 85 Flo H.:

When running Fedora 28, one can expect to have packages from other releases, likely 26, 27, and maybe even 29, no?

Comment 86 Kalev Lember:

Hi Flo! Yes, should be fine to delete all of them. We put .fc25, .fc26, .fc27 etc packages that belong to Fedora 28 to /var/cache/PackageKit/28 and don't load anything from the older directories.

Me:

My script "rpm-cache-prune", attached to this bz since July, deletes all rpms in the cache that are not currently installed.

On my F27 system now, it leaves nothing in previous release versions directories.

BUT that isn't the case, generally speaking.  I've quite often seen packages from previous releases remain installed.

SO: the advice is correct now, for a fully upgraded F27, as far as I know.  But it isn't generally true.

See next message for a safe process.

Comment 90 D. Hugh Redelmeier 2018-03-09 14:47:37 UTC
My script "rpm-cache-prune", attached to this bz since July, deletes all rpms in the cache that are not currently installed.

If you want quick relief, I suggest:
1. [optional] apply updates
2. [needed if you've updated since the last reboot] reboot (to make sure that you are running only what is installed)
3. run rpm-cache-prune script

If you are actually out of space, you don't want to do step 1.  Operating a UNIX-family system with a full filesystem invites many kinds of peculiar problems.

Unfortunately you have to run this script whenever you want to reclaim space.  It is a stopgap.

Comment 91 Hirm Loh 2018-03-10 14:33:17 UTC
sudo find /var/cache/PackageKit/ -name "*.rpm" -delete

nothing work on this command, when i run, or i should run another command ?


but i have asked about disk /var is full on fedorforum, some guys gave me commands , check this --> https://forums.fedoraforum.org/showthread.php?317508-var-is-almost-full/page2

Comment 92 Hirm Loh 2018-03-10 14:34:46 UTC
sudo find /var/cache/PackageKit/ -name "*.rpm" -delete

nothing work on this command, when i run, or i should run another command ?


but i have asked about disk /var is full on fedorforum, some guys gave me commands , check this --> https://forums.fedoraforum.org/showthread.php?317508-var-is-almost-full/page2

Comment 93 Fedora Update System 2018-03-16 16:12:41 UTC
PackageKit-1.1.9-2.fc27 gnome-software-3.28.0-4.fc27 libappstream-glib-0.7.7-2.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2018-55a6726164

Comment 94 Kalev Lember 2018-03-16 16:19:10 UTC
*** Bug 1528757 has been marked as a duplicate of this bug. ***

Comment 95 Fedora Update System 2018-03-17 20:38:26 UTC
PackageKit-1.1.9-2.fc27, gnome-software-3.28.0-4.fc27, libappstream-glib-0.7.7-2.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-55a6726164

Comment 96 Fedora Update System 2018-04-13 15:55:49 UTC
PackageKit-1.1.9-3.fc27 gnome-software-3.28.1-1.fc27 libappstream-glib-0.7.7-2.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2018-55a6726164

Comment 97 Fedora Update System 2018-04-15 16:18:57 UTC
PackageKit-1.1.9-3.fc27, gnome-software-3.28.1-1.fc27, libappstream-glib-0.7.7-2.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-55a6726164

Comment 98 Fedora Update System 2018-04-19 00:29:26 UTC
PackageKit-1.1.9-3.fc27, gnome-software-3.28.1-1.fc27, libappstream-glib-0.7.7-2.fc27 has been pushed to the Fedora 27 stable repository. If problems still persist, please make note of it in this bug report.

Comment 99 Rex Dieter 2018-04-30 13:28:13 UTC
*** Bug 1573039 has been marked as a duplicate of this bug. ***

Comment 100 David Binderman 2018-05-11 08:16:13 UTC
Packagekit daemon still runs in fedora 28. What a pain !

There are two parts to this bug: 

1. use of bandwidth to download rpms
2. storing the rpms after download.

#2 may or may not be fixed. See above.
#1 can maybe be fixed by this command as root:
 
systemctl mask packagekit.service

Comment 101 Fabio Olive Leite 2018-05-11 12:50:11 UTC
(In reply to David Binderman from comment #100)
> 1. use of bandwidth to download rpms
> #1 can maybe be fixed by this command as root:
> systemctl mask packagekit.service

You can also run the command below, as mentioned by Thomas Mueller in #17 and Chris Murphy in #68.

gsettings set org.gnome.software download-updates false

Comment 102 Brian J. Murrell 2018-05-11 16:42:40 UTC
I have PackageKit-1.1.10-1.fc27.x86_64 installed.

The space usage:

$ du -sh /var/cache/PackageKit/27/metadata/updates/packages
3.1G	/var/cache/PackageKit/27/metadata/updates/packages

and the package duplication:

-rw-r--r--. 1 root root  1542224 Feb 13 06:45 vim-X11-8.0.1478-1.fc27.x86_64.rpm
-rw-r--r--. 1 root root  1544336 Feb 15 06:51 vim-X11-8.0.1505-1.fc27.x86_64.rpm
-rw-r--r--. 1 root root  1548944 Feb 21 06:48 vim-X11-8.0.1523-1.fc27.x86_64.rpm
-rw-r--r--. 1 root root  1549052 Feb 24 06:33 vim-X11-8.0.1527-1.fc27.x86_64.rpm
-rw-r--r--. 1 root root  1550364 Mar  2 06:33 vim-X11-8.0.1553-1.fc27.x86_64.rpm
-rw-r--r--. 1 root root  1552680 Mar 12 06:14 vim-X11-8.0.1573-1.fc27.x86_64.rpm
-rw-r--r--. 1 root root  1558864 Apr 11 06:14 vim-X11-8.0.1666-1.fc27.x86_64.rpm
-rw-r--r--. 1 root root  1560436 Apr 18 07:02 vim-X11-8.0.1704-1.fc27.x86_64.rpm
-rw-r--r--. 1 root root  1561928 May  1 06:25 vim-X11-8.0.1763-1.fc27.x86_64.rpm

I don't even use GNOME Software to update.  I use DNF and it's automatic updater.  I guess I should just follow the:

> gsettings set org.gnome.software download-updates false

advise and write GNOME Software off as untenable.

Comment 103 Steeve McCauley 2018-07-05 15:30:08 UTC
An absurd situation to have dnf and PackageKit collide with caching of packages, even more absurd that this is marked as an ERRATA and CLOSED.  And the use of offline updates by PackageKit is the kind of thing that keeps me from using Windows or MacOS.

Not sure if gsettings download-updates has to be disabled for all users, probably all admins at least, but this would seem to be the best approach to removing use of PackageKit for those who are updating with dnf.

# not sure if this has to be done for all users
gsettings set org.gnome.software download-updates false
# turn off PackageKit
sudo systemctl mask packagekit.service
# clear current OS versions PackageKit cache for packages already installed
sudo pkcon refresh force

# optional
sudo rm -rf /var/cache/PackageKit

Comment 104 basilicum 2018-07-05 18:43:10 UTC
I totally agree with previous comment.

Get rid of Packagekit as it relies on unfinished software:
https://www.ctrl.blog/entry/packagekit-dnf

Use dnf automated instead:
https://www.ctrl.blog/entry/how-to-dnf-automatic

Very strong advice to Fedora to remove packagekit form the distro

Comment 105 David Demelier 2018-08-09 16:13:51 UTC
Can you please reopen this bug as this issue is *not* fixed? PackageKit is definitely the best way to kill a SSD. Can't believe it's still not fixed.

Comment 106 Standa Laznicka 2018-08-10 06:27:58 UTC
Since noone from the PackageKit component reopened it and it still seems to bug a lot of people, I am reopening the BZ.

Also adding needinfo for the assignee to provide insight on the issue and possibly explain why it was/is considered fixed.

Comment 107 D. Hugh Redelmeier 2018-08-10 20:40:33 UTC
I don't seem to have the problem any longer (Fedora 28).  When I run my "rpm-cache-prune" script (see above), it no longer finds gobs of RPMs to remove.

Installed RPMs are still retained.  I think that this is a feature, not a bug.  Those are probably used for the DRPM feature (guess).

Other people might have different experiences.

Comment 108 Matthew Miller 2018-08-29 14:37:46 UTC
For the record from the Bodhi description of the update (linked above):

>  PackageKit 1.1.9 adds basic cache cleanup, removing old cache directories for previous release versions (24, 25, 26 etc) when running on Fedora 27 -- something that people have been asking for a while. (There's more cache cleanup fixes to follow in the future, this is just a first step.)

So, this covers the "ancient releases still taking space" case, but not the "superseded updates persist forever" one. I think we probably should leave this open until the further work happens, but it's good to note that progress is made.

Comment 109 Ben Cotton 2018-08-29 15:12:12 UTC
In today's Prioritized Bugs and Issues meeting, this was accepted as a Prioritized Bug.

https://meetbot.fedoraproject.org/fedora-meeting-2/2018-08-29/fedora_prioritized_bugs_and_issues.2018-08-29-14.00.log.html#l-138

Comment 110 D. Hugh Redelmeier 2018-10-04 17:33:22 UTC
In 107 I said that the problem seemed to be fixed for my F28 systems (all I have).

One of those machines started complaining about low space on / a couple of days ago.  So I ran my "rpm-cache-prune" script (attached to this bz for over a year).  It found a tonne of obsolete packages on each machine.

I don't know why the bug seemed to go away and then come back.

Comment 111 Steve Susbauer 2018-11-08 05:57:13 UTC
It is possible to disable updates in Gnome software, which I believe will allow you to keep PackageKit around for GUI software installs without the update cache issue. Would need to continue updating through dnf.

gsettings set org.gnome.software allow-updates false

Comment 112 Federico Bruni 2018-11-08 08:54:49 UTC
Steve, I think that allow-update (whether Software should handle upgrades) is the wrong key to work around this problem.
I have set it to true, but I don't have this problem. I think you want to set download-updates (whether to automatically download updates) to false:

$ gsettings get org.gnome.software download-updates
false

Here's my current configuration on my desktop machine, still running Fedora 28.

$ gsettings get org.gnome.software allow-updates
true

$ gsettings get org.gnome.software download-updates
false

$ cat /etc/PackageKit/PackageKit.conf | grep KeepCache
#KeepCache=false

I always upgrade the system using dnf. However, my PackageKit cache is quite lightweight:

/var/cache/PackageKit/28 = 210 MB
/var/cache/PackageKit/29 = 180 MB

I'm running PackageKit version 1.1.10.

Comment 113 Kalev Lember 2018-11-23 17:13:58 UTC
OK, th(In reply to Matthew Miller from comment #108)
> For the record from the Bodhi description of the update (linked above):
> 
> >  PackageKit 1.1.9 adds basic cache cleanup, removing old cache directories for previous release versions (24, 25, 26 etc) when running on Fedora 27 -- something that people have been asking for a while. (There's more cache cleanup fixes to follow in the future, this is just a first step.)
> 
> So, this covers the "ancient releases still taking space" case, but not the
> "superseded updates persist forever" one. I think we probably should leave
> this open until the further work happens, but it's good to note that
> progress is made.

OK, the "superseded updates persist forever" issue should now be fixed as well with https://github.com/hughsie/PackageKit/commit/66f2966fb6484a86324f98302659c5f27a02b1b5

Comment 114 Fedora Update System 2018-11-28 09:00:54 UTC
PackageKit-1.1.12-1.fc29 has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2018-c3dd0383fa

Comment 115 Fedora Update System 2018-11-29 03:16:42 UTC
PackageKit-1.1.12-1.fc29 has been pushed to the Fedora 29 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-c3dd0383fa

Comment 116 Fedora Update System 2018-12-07 02:38:09 UTC
PackageKit-1.1.12-1.fc29 has been pushed to the Fedora 29 stable repository. If problems still persist, please make note of it in this bug report.

Comment 117 mlaverdiere 2019-11-10 15:49:58 UTC
This bug doesn't seem to be fixed yet. I'm on Fedora 31 and I still see rpm packages accumulating in the /var/cache/PackageKit/metadata/updates/packages/

The "out of the box" PackageKit.conf files is still set up to keep rpm packages after download:

# Keep the packages after they have been downloaded
#KeepCache=false

Should I open another report for F31?

Comment 118 Michael Catanzaro 2019-11-10 17:37:37 UTC
Let's just reopen this one.

Comment 119 Kalev Lember 2019-11-11 08:14:14 UTC
This was implemented a few releases ago. Please don't reopen old bugs (and email 110 people in the process).

If you think you have a bug with rpms accumulating under packagekit cache, please file a new bug. What I suspect is going on here is that gnome-software is simply preparing offline updates and gathering rpms because of that. This is completely normal.

Comment 120 mlaverdiere 2019-11-11 12:42:41 UTC
What was implemented? 

I'm on a freshly installed Fedora 31 system and what I see is this: gnome-software is still downloading rpms and keeping them forever (under /var/cache/PackageKit/metadata) since I do upgrades with dnf.

So unless I missed something, the main point of this bug report seems to still be valid, i.e. PackageKit accumulating RPMs packages in /var/cache/PackageKit/metadata and filling root filesystem with unused RPM files.

Comment 122 Steeve McCauley 2019-11-11 13:47:31 UTC
The point of this bug was the fact that PackageKit was never clearing its cache if rpms weren't installed via gnome-software (or pkcon).  If you update with dnf but PackageKit has already downloaded an update it just sits in the cache until the same rpm is updated again, at which point it will finally be flushed.  It would be nice if dnf could be configured to get look for updates from a local PackageKit cache, but that's a different feature and would probably require collaboration between two competing projects.

Comment 123 mlaverdiere 2019-11-11 15:28:49 UTC
So, with the explanations provided by Kalev and Steeve, am I right to think that there has been some progress on this bug, but not to a point where it would prevent PackageKit to accumulate a load of RPM packages in /var/cache/PackageKit/metadata and fill the root filesystem?

According to the following commit and Steeve's explanation, the PackageKit cache cleanup would only occur when a new RPM for the same package is available, which is a progress, but maybe not enough to prevent disk saturation with useless RPM: https://github.com/hughsie/PackageKit/commit/66f2966fb6484a86324f98302659c5f27a02b1b5

So if that is true, should we just reopen this but report or create a new one?

For the record and to summarize what was said before, here's what I found to be a useful workaround on my system:

1. Setting up PackageKit not to download any updates automatically, by issuing this command in a terminal:

gsettings set org.gnome.software download-updates false 

2. Setting up PackageKit not to cache packages at all (not sure if it is sill effective, according to this: https://unix.stackexchange.com/questions/265755/fedora-23-can-i-safely-delete-files-in-var-cache-packagekit-metadata-updates/283556):

In the file /etc/PackageKit/PackageKit.conf, uncomment #KeepCache=false (so it should read:  KeepCache=false)

Comment 124 Michael Catanzaro 2019-11-11 17:26:24 UTC
Apologies for reopening this.

> So if that is true, should we just reopen this but report or create a new one?

Yes, please create a new bug.

Comment 125 cornel panceac 2019-11-12 05:45:50 UTC
(In reply to Steeve McCauley from comment #122)
> The point of this bug was the fact that PackageKit was never clearing its
> cache if rpms weren't installed via gnome-software (or pkcon).  If you
> update with dnf but PackageKit has already downloaded an update it just sits
> in the cache until the same rpm is updated again, at which point it will
> finally be flushed.  It would be nice if dnf could be configured to get look
> for updates from a local PackageKit cache, but that's a different feature
> and would probably require collaboration between two competing projects.

As far as i understand, the "point of this bug" was:


"PackageKit should not keep around .rpm files needlessly (eg. older or up to date packages seem worthless) and/or should provide a mean to erase its cache."

, as can be seen in the first post of its creator.

If any of these two things still happen, i consider the bug is still present.

If we still feel like reopening a new ticket, please at least mention this old one so that the original knowledge is not disconnected.

Comment 126 mlaverdiere 2019-11-16 13:24:15 UTC
New bug report filed:https://bugzilla.redhat.com/show_bug.cgi?id=1773202


Note You need to log in before you can comment on or make changes to this bug.