Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1091702 - [rfe] complete-transaction support
Summary: [rfe] complete-transaction support
Keywords:
Status: NEW
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf
Version: rawhide
Hardware: Unspecified
OS: Unspecified
medium
unspecified
Target Milestone: ---
Assignee: Packaging Maintenance Team
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1242135 (view as bug list)
Depends On:
Blocks: dnf-community 1723126
TreeView+ depends on / blocked
 
Reported: 2014-04-27 12:25 UTC by Susi Lehtola
Modified: 2023-08-09 19:50 UTC (History)
50 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-11-27 14:16:14 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1382063 0 low CLOSED RFE - Expand Upgrade function to optionally perform "offline" upgrades 2021-02-22 00:41:40 UTC
Red Hat Bugzilla 1382224 0 medium CLOSED RFE: dnf transactions should run in a transient systemd service 2021-02-22 00:41:40 UTC

Internal Links: 1382063 1382224

Description Susi Lehtola 2014-04-27 12:25:19 UTC
It would be useful to have an analogous tool to yum-complete-transaction in dnf.

Sometimes it happens e.g. that the battery runs out while installing updates to a laptop, which is then pretty painstaking to fix without an automatic tool.

Comment 1 Ales Kozumplik 2014-04-28 05:57:32 UTC
I am reluctant about adding complete-transaction again, because DNF can not control what happens with the RPMDB until the complete-transaction is run again and faces applying outdated transaction to the system (dependencies have gone missing, packages depending on now duplicate versions might have been installed etc.) Closign this as WONTFIX with a vote, i.e. once enough people are CCed in the bug, we will see about adding this.

Note that the problem of unfinished transaction generally means two different things:

a) transaction was terminated during the install phase and some packages are left to be installed. In this case, running 'dnf upgrade' again will do the right thing and install the remaining packages.

b) transaction was terminated during the erase phase. Some packages are left to be installed. In this case the duplicate versions of packages will be removed on their next upgrade. Alternatively, the autoremove plugin should be eventually capable of removing duplicate, non-installonly packages from the system (bug 963345).

Comment 2 Honza Silhan 2015-07-29 14:01:24 UTC
*** Bug 1242135 has been marked as a duplicate of this bug. ***

Comment 3 Mathieu Bridon 2015-09-09 17:36:40 UTC
> a) transaction was terminated during the install phase and some packages
> are left to be installed. In this case, running 'dnf upgrade' again will
> do the right thing and install the remaining packages.

Really?

Running "dnf upgrade" will install the remaining packages... even if the transaction which had failed was a "dnf install foo bar baz"?

If that's the case, then at the very least, please add a message about that (similarly to how yum used to ask us to run yum-complete-transaction), because I (and I suspect others) would have never figured that I could finish an **install** transaction by running a new **upgrade**.

Comment 4 Vladimir Stackov 2016-01-28 19:11:35 UTC
(In reply to Ales Kozumplik from comment #1)
> b) transaction was terminated during the erase phase. Some packages are left
> to be installed. In this case the duplicate versions of packages will be
> removed on their next upgrade. Alternatively, the autoremove plugin should
> be eventually capable of removing duplicate, non-installonly packages from
> the system (bug 963345).

In this case I'm still forced to use package-cleanup or clean by hands because autoremove won't help.

Comment 5 Miroslav Suchý 2016-06-30 21:04:37 UTC
I have been upgrading from F23 -> F24 and my notebook run out of power and switched off. The transaction was not finished.
"package-cleanup --cleandupes" done the job. But ordinary user has very little chance to identify whether he fit in a) or b) and what is the right action to do.

Comment 6 Kevin Kofler 2016-10-05 16:56:22 UTC
This is a vital feature to recover from crashes during the upgrade process. And of course people will only notice that it is missing when they actually have a messed up system! So user demand is almost certainly much higher than what you can see in this bug.

Comment 7 Kevin Kofler 2016-10-05 17:00:56 UTC
In the unified DNF 2 world, the accounting needs to be handled at libdnf level so that PackageKit-dnf transactions will also be protected.

Comment 8 Adam Williamson 2016-10-06 04:20:27 UTC
I agree that it would be a sensible improvement to have this feature. The issue with completing 'outdated' transactions doesn't seem significant enough to block this feature, to me.

It's really a big problem if you manage to get an incomplete update transaction. There's no very clean way for a user to recover. package-cleanup doesn't really do it; it can probably get your database back to a 'clean' state but it can't do anything about the missed scriptlets (and maybe there are other things dnf does in the missed phases that package-cleanup can't cover, I dunno).

Obviously we could refuse to (or warn hard about) completing a transaction which isn't the most recent transaction. As for non-DNF transactions, well, I don't know enough about the internals to make concrete suggestions, but some thoughts:

1. If we can tell the *time* of the last modification to the RPM database, can't we just check if it's after the aborted transaction failed?

2. For each package in the aborted transaction, we could check if the RPM database reports any EVR other than one that was involved in the aborted transaction, and abort / warn in that case?

The need for this feature is of course correspondingly greater so long as https://bugzilla.redhat.com/show_bug.cgi?id=1382063 and/or https://bugzilla.redhat.com/show_bug.cgi?id=1382224 (or similar) are not implemented.

Comment 9 Hedayat Vatankhah 2016-12-10 06:51:44 UTC
There is always the possibility of loss of power (or other hardware failure) during dnf operations. So, having complete-transaction is always needed.

Comment 10 Matthew Miller 2016-12-15 00:52:30 UTC
Could the DNF team please re-evaluate this, in the light of the comments above? Thanks!

Comment 11 Honza Silhan 2016-12-19 13:49:05 UTC
ok, reconsidered. The new plugin may call passively removal of duplicates (bug 1284981) + fix other easily detected problems. The solution will not be 100% bullet proof though.

Comment 12 Matthew Miller 2016-12-19 13:51:16 UTC
Thanks! I don't think bulletproof is expected, but hopefully we can provide something in at least some situations which would otherwise be painful.

Comment 13 Kevin Kofler 2016-12-19 18:09:21 UTC
It's best to take into account what the last, failed, transaction was.

Comment 14 Hedayat Vatankhah 2016-12-20 06:54:29 UTC
(In reply to Honza Silhan from comment #11)
> ok, reconsidered. The new plugin may call passively removal of duplicates
> (bug 1284981) + fix other easily detected problems. The solution will not be
> 100% bullet proof though.

How would you remove duplicates? When I use 'advised' methods (like the one in #1284981), it doesn't work as it tries to remove lots of packages as 'dependencies'. So, each time I was in such a situation, the only way out of it for me was to 'rpm -e --nodeps OLD_VER_PACKAGES'. I didn't find ANY way to deal with such cases using dnf alone. I agree with Kevin that it should try to finish the last transaction.

Comment 15 Zdenek Kabelac 2017-08-22 12:39:53 UTC
So - yesterday during  mass-rebuild upgrade of rawhide I've 'witnessed' again kill of 'dnf' in the middle of upgrade  of  4000 packages  (thankfully in cleanup phase)

But I've been left with 1000 duplicate packages.

It's quite annoying manual work to clean-up this mess left after 'dnf' being aborted.

I'm amazed why there is not at least a SIMPLE reporting tool showing me older packages for removal in year 2017....

It should be even built-in  'rpm' functionality - since  RPM simply HAS to know which of the duplicated package is the file owner - eventually marking packages that needs full reinstall.

Something as simple as    'rpm --resolve-duplicates' should be there for years - rpm DB has ALL the info....

Also note:

dnf  repoquery  --duplicated  --latest-limit -1 

No longer works!  '-1' is not an accepted argument - even though it's even printed in --help....

Comment 16 Joseph D. Wagner 2017-08-22 15:32:22 UTC
Last I heard, Lennart Poettering was working on systemd-rpm, which should take care of it. :D

Comment 17 Mikhail 2017-08-22 15:35:03 UTC
If you on Rawhide you can remove all duplicate packages with command:
# dnf remove --duplicates --setopt=protected_packages=

Comment 18 Parag Nemade 2017-08-23 06:19:25 UTC
I too got the same problem when I updated f26 to rawhide last week, I managed to remove many duplicates with 
dnf distro-sync --releasever=rawhide

but now left with sudo and systemd-* duplicate packages. I am not able to find yet a way to remove these duplicate packages.

Anyone knows how to remove duplicates of sudo and systemd* packages?

Comment 19 Parag Nemade 2017-08-23 06:21:20 UTC
(In reply to Mikhail from comment #17)
> If you on Rawhide you can remove all duplicate packages with command:
> # dnf remove --duplicates --setopt=protected_packages=

ah I missed to see this comment. I just used this command and it removed sudo and systemd duplicate packages.

Thanks Mikhail.

Comment 20 Richard Shaw 2017-09-13 19:44:49 UTC
(In reply to Mikhail from comment #17)
> If you on Rawhide you can remove all duplicate packages with command:
> # dnf remove --duplicates --setopt=protected_packages=

I just tried this on a f26 system which had the transaction interrupted because it couldn't find a package which was on an NFS share.

Now both rpm and dnf have been removed so I'm basically screwed.

Comment 21 Matthew Miller 2017-09-13 19:57:51 UTC
Ouch. You should be able to repair that by booting with a rescue CD and using `rpm --root`. 

But: that shouldn't happen anyway, as "--duplicates" should only remove the older versions and leave newer versions intact. There might be a (separate) bug there. :(

Comment 22 Richard Shaw 2017-09-13 20:09:13 UTC
(In reply to Matthew Miller from comment #21)
> Ouch. You should be able to repair that by booting with a rescue CD and
> using `rpm --root`. 

That's pretty much the plan but I was trying to fix it remotely, now I have to wait until I can get in front of it. dnf has the --installroot option so that should work I'm guessing. About 500+ packages were removed. Should I try group installing "Workstation" to make sure I have a basic functioning system?

 
> But: that shouldn't happen anyway, as "--duplicates" should only remove the
> older versions and leave newer versions intact. There might be a (separate)
> bug there. :(

Not sure I want to reproduce the problem to verify it :)

Comment 23 Ben Cotton 2019-11-27 14:16:14 UTC
Fedora 29 changed to end-of-life (EOL) status on 2019-11-26. Fedora 29 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 24 Ben Cotton 2019-11-27 14:37:08 UTC
This bug was accidentally closed due to a query error. Reopening.

Comment 25 Kevin Fenzi 2020-10-31 20:01:56 UTC
I think this can be closed now?

       "dnf history replay [--ignore-installed] [--ignore-extras] [--skip-unavailable] <filename>
              Replay  a  transaction  stored in file <filename> by History Store Command. The replay will perform
              the exact same operations on the packages as in the original transaction and will  return  with  an
              error  if case of any differences in installed packages or their versions. See also the Transaction
              JSON Format specification of the file format.

              --ignore-installed
                     Don't check for the installed packages being in the same state  as  those  recorded  in  the
                     transaction.  E.g. in case there is an upgrade foo-1.0 -> foo-2.0 stored in the transaction,
                     but there is foo-1.1 installed on the target system.

              --ignore-extras
                     Don't check for extra packages pulled into the transaction on the target  system.  E.g.  the
                     target  system  may  not have some dependency, which was installed on the source system. The
                     replay errors out on this by default, as the transaction would not be the same.

              --skip-unavailable
                     In case some packages stored in the transaction are not available on the target system, skip
                     them instead of erroring out."

Comment 26 Kevin Kofler 2020-10-31 22:43:19 UTC
No, replaying is not the same as completing an interrupted transaction. As hinted by the description ("and will return with an error if case of any differences in installed packages or their versions"), replaying expects to find the same initial state as before the transaction, whereas completing needs to handle any intermediate states. Sure, you can skip the main sanity check with --ignore-installed, but I doubt that "dnf history replay" will know what to do with an interrupted transaction that left more than one version of the same package installed (or at least, marked as such in the RPM database), which is the main error for whose resolution we need complete-transaction support. (It is the most common case when a transaction gets interrupted in the middle.)

Comment 27 Sam Varshavchik 2020-11-01 01:03:18 UTC
Furthermore, how would I know what the <filename> is? Where does that come from? I started a "dnf upgrade", and for whatever reason there was a crash or a reboot. And I want to clean that up. I didn't have any <filename> to specify, to "dnf upgrade", and I don't have one now.

At least some time ago there were some reports (including from me) sporadic segfaults after dnf downloaded what it was going to install. In all cases the segfault happened before the actual transaction started, but something in the download code was corrupting memory. Haven't happened in a while, so the bug is probably fixed, I guess. But in any case, something blew up, and I just want to restart it and complete what wasn't completed, without knowing where some <filename> is.

Comment 28 Kevin Fenzi 2020-11-01 01:18:04 UTC
(In reply to Kevin Kofler from comment #26)
> No, replaying is not the same as completing an interrupted transaction. As
> hinted by the description ("and will return with an error if case of any
> differences in installed packages or their versions"), replaying expects to
> find the same initial state as before the transaction, whereas completing
> needs to handle any intermediate states. Sure, you can skip the main sanity
> check with --ignore-installed, but I doubt that "dnf history replay" will
> know what to do with an interrupted transaction that left more than one
> version of the same package installed (or at least, marked as such in the
> RPM database), which is the main error for whose resolution we need
> complete-transaction support. (It is the most common case when a transaction
> gets interrupted in the middle.)

You doubt, but haven't tested? ok. I do think it will handle the case, but I also haven't tested it. 

(In reply to Sam Varshavchik from comment #27)
> Furthermore, how would I know what the <filename> is? Where does that come
> from? I started a "dnf upgrade", and for whatever reason there was a crash
> or a reboot. And I want to clean that up. I didn't have any <filename> to
> specify, to "dnf upgrade", and I don't have one now.

You need to create it with 'dnf history store N > filename' specifying the history entry you want to store into a firename.
> 
> At least some time ago there were some reports (including from me) sporadic
> segfaults after dnf downloaded what it was going to install. In all cases
> the segfault happened before the actual transaction started, but something
> in the download code was corrupting memory. Haven't happened in a while, so
> the bug is probably fixed, I guess. But in any case, something blew up, and
> I just want to restart it and complete what wasn't completed, without
> knowing where some <filename> is.

Yyou can do 'dnf history store N > filename' and then replay that, possibly editing the json history file to adjust with what you want to do. 

In any case, likely this will be my last reply for now... I am not a dnf maintainer, I simply found this and thought it might meet the needs requested in this bug. 
If not, oh well.

Comment 29 Kevin Kofler 2020-11-01 10:17:40 UTC
Well, the replay command clearly was not designed for this use case. If it works at all, it would be purely by accident, and you have to jump through hoops to use it: it should not be necessary to store the history entry to a file and then to replay it, that's what "dnf history redo last" is supposed to be for, but "redo" does not have the flags to skip the sanity checks.

It is still an absolute requirement to have a complete-transaction command designed for this use case and tested (!) with this use case. I do not understand why such an essential requirement gets ignored for more than 6 years, whereas DNF developers get tasked by Red Hat management to work on unnecessary features such as modularity instead.

Comment 30 Kevin Kofler 2020-11-01 10:19:04 UTC
Oh, and does an interrupted command even show up in the history at all, so you can store and replay it from there? If the history entry is created only after the transaction is complete, then there is nothing you can replay.

Comment 31 Richard Shaw 2020-11-01 15:28:27 UTC
I may try setting up a VM to simulate a reboot/power outage but I agree with Kevin, even if it works it's extremely cumbersome and unintuitive.

Comment 32 Graham Leggett 2022-01-23 11:47:45 UTC
Just had a system die halfway through a rawhide upgrade.

Am I right in understand that dnf has regressed to no longer supporting the continuation of a transaction compared to yum?

Comment 33 Richard Shaw 2022-01-23 14:00:11 UTC
Not a regression in dnf per se. as it never had it but could be viewed as a regression from yum.

Comment 34 Nathaniel W. Turner 2022-04-23 03:20:26 UTC
Is the problem here that while yum was transactional, dnf is not? 

Or is dnf transactional in theory, but there is just no way to actually test that transactions can be rolled forward to a consistent state if they're interrupted?

Comment 35 Graham Leggett 2022-06-14 14:26:46 UTC
Another day, another system dies, this time in the middle of a dnf update because of a kernel panic.

This time we have RPM provided files of zero length, it looks like dnf isn't syncing the disk during upgrades either.

This is a significant regression.

Comment 36 John 2023-03-02 23:50:35 UTC
When you see the response to bugs like this, it really gives you an insight into the actual mindset that Red Hat have, as far as delivering a quality product.

A bug like this should get some priority, but... nope.

Red Hat just does not really care about quality & reliability at all.

Comment 37 Jaroslav Mracek 2023-08-09 19:50:22 UTC
There are some improvements in dnf. It is possible to remove duplicates and reinstall remaining  package to ensure its functionality - just use `dnf remove --duplicates`. In many cases that simple operation fails on broken systems, but in this cases it is only possible to resolve the issue using directly RPM.

I agree that we need additional improvements.


Note You need to log in before you can comment on or make changes to this bug.