Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1596827 - DNF 3 crashes with "UNIQUE constraint failed" for comps_environment_group.groupid or trans_with.trans_id
Summary: DNF 3 crashes with "UNIQUE constraint failed" for comps_environment_group.gro...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf
Version: 29
Hardware: x86_64
OS: Linux
unspecified
urgent
Target Milestone: ---
Assignee: Daniel Mach
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: AcceptedBlocker https://fedoraproject...
: 1598590 1624682 1634305 (view as bug list)
Depends On:
Blocks: F29FinalBlocker 1628844
TreeView+ depends on / blocked
 
Reported: 2018-06-29 18:51 UTC by George R. Goffe
Modified: 2018-10-07 20:59 UTC (History)
17 users (show)

Fixed In Version: dnf-3.6.1-1.fc29
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-10-07 20:59:23 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
gzip'd execution log of the beginning of this problem (7.78 KB, application/x-gzip)
2018-06-29 18:51 UTC, George R. Goffe
no flags Details
tar gzip'd file containing console output from failure (1.21 KB, application/x-gzip)
2018-08-03 04:31 UTC, George R. Goffe
no flags Details

Description George R. Goffe 2018-06-29 18:51:56 UTC
Created attachment 1455552 [details]
gzip'd execution log of the beginning of this problem

Description of problem:
after attempting to upgrade this system with this command "dnf -y --best upgrade --allowerasing dnf-data.noarch dnf-plugins-core.noarch dnf.noarch", dnf became totally unusable. See enclosed log below.


Version-Release number of selected component (if applicable):
dnf-3.0.1-1.fc29.noarch

How reproducible:
always

Steps to Reproduce:
1.see above
2.
3.

Actual results:
TOTAL catastrophe

Expected results:
Successful and clean system upgrade.

Additional info:

dnf -y --best upgrade --allowerasing vala.x86_64
Fedora - Rawhide - Developmental packages for the next Fedora release                                                 21 kB/s |  16 kB     00:00    
Last metadata expiration check: 0:00:00 ago on Fri 29 Jun 2018 11:44:56 AM PDT.
Traceback (most recent call last):
  File "/bin/dnf", line 58, in <module>
    main.user_main(sys.argv[1:], exit_code=True)
  File "/usr/lib/python3.6/site-packages/dnf/cli/main.py", line 179, in user_main
    errcode = main(args)
  File "/usr/lib/python3.6/site-packages/dnf/cli/main.py", line 64, in main
    return _main(base, args, cli_class, option_parser_class)
  File "/usr/lib/python3.6/site-packages/dnf/cli/main.py", line 99, in _main
    return cli_run(cli, base)
  File "/usr/lib/python3.6/site-packages/dnf/cli/main.py", line 123, in cli_run
    ret = resolving(cli, base)
  File "/usr/lib/python3.6/site-packages/dnf/cli/main.py", line 146, in resolving
    base.resolve(cli.demands.allow_erasing)
  File "/usr/lib/python3.6/site-packages/dnf/base.py", line 817, in resolve
    self._transaction = self._goal2transaction(goal)
  File "/usr/lib/python3.6/site-packages/dnf/base.py", line 737, in _goal2transaction
    ts.add_upgrade(pkg, upgraded[0], obs)
  File "/usr/lib/python3.6/site-packages/dnf/db/group.py", line 269, in add_upgrade
    ti_new = self.new(new, libdnf.transaction.TransactionItemAction_UPGRADE)
  File "/usr/lib/python3.6/site-packages/dnf/db/group.py", line 219, in new
    rpm_item = self._pkg_to_swdb_rpm_item(pkg)
  File "/usr/lib/python3.6/site-packages/dnf/db/group.py", line 210, in _pkg_to_swdb_rpm_item
    rpm_item = self.history.swdb.createRPMItem()
  File "/usr/lib/python3.6/site-packages/dnf/db/history.py", line 290, in swdb
    self._swdb = libdnf.transaction.Swdb(self.dbpath)
  File "/usr/lib64/python3.6/site-packages/libdnf/transaction.py", line 713, in __init__
    this = _transaction.new_Swdb(*args)
RuntimeError: C++ std::exception: Step: UNIQUE constraint failed: comps_environment_group.environment_id, comps_environment_group.groupid in

        INSERT INTO
            comps_environment_group (
                environment_id,
                groupid,
                installed,
                group_type
            )
        VALUES
            (144536, 'libreoffice', 1, 4)

Comment 1 Sunil P 2018-07-03 13:59:56 UTC
My rawhide installation is also broken with similar error message.

$ sudo dnf history
Traceback (most recent call last):
  File "/bin/dnf", line 58, in <module>
    main.user_main(sys.argv[1:], exit_code=True)
  File "/usr/lib/python3.6/site-packages/dnf/cli/main.py", line 179, in user_main
    errcode = main(args)
  File "/usr/lib/python3.6/site-packages/dnf/cli/main.py", line 64, in main
    return _main(base, args, cli_class, option_parser_class)
  File "/usr/lib/python3.6/site-packages/dnf/cli/main.py", line 95, in _main
    cli.configure(list(map(ucd, args)), option_parser())
  File "/usr/lib/python3.6/site-packages/dnf/cli/cli.py", line 905, in configure
    self.command.configure()
  File "/usr/lib/python3.6/site-packages/dnf/cli/commands/__init__.py", line 849, in configure
    if not os.access(self.base.history.path, os.R_OK):
  File "/usr/lib/python3.6/site-packages/dnf/db/history.py", line 308, in path
    return self.swdb.getPath()
  File "/usr/lib/python3.6/site-packages/dnf/db/history.py", line 290, in swdb
    self._swdb = libdnf.transaction.Swdb(self.dbpath)
  File "/usr/lib64/python3.6/site-packages/libdnf/transaction.py", line 713, in __init__
    this = _transaction.new_Swdb(*args)
RuntimeError: C++ std::exception: Step: UNIQUE constraint failed: trans_with.trans_id, trans_with.item_id in

            INSERT INTO
                trans_with (
                    trans_id,
                    item_id
                )
            VALUES
                (3777, 20073)

Comment 2 Mattias Ellert 2018-07-04 07:57:43 UTC
With this version of dnf --

# rpm -q dnf
dnf-3.0.1-1.fc29.noarch

-- updates are not possible. Dnf can not parse the configuration files in /etc/yum.conf.d any more.

# dnf update
Förrådet EGI-trustanchors har ingen spegel eller bas-url satt.
Förrådet rawhide har ingen spegel eller bas-url satt.
Förrådet rawhide-debuginfo har ingen spegel eller bas-url satt.
Förrådet rawhide-source har ingen spegel eller bas-url satt.
Fel: Det finns inga aktiva förråd.

It says "no mirror or base-url set". I tried to set LANG to C to get the error message in English for the purpose of this comment, but then dnf suddenly can parse the configuration. However, now it segfaults instead.

# LANG=C.UTF-8 dnf update
Last metadata expiration check: 0:43:42 ago on Wed Jul  4 09:08:21 2018.
Segmenteringsfel (minnesutskrift skapad)

Comment 3 Adam Williamson 2018-07-04 17:40:46 UTC
Mattias: those both look like different problems, so can you please file separate bugs for them? A single huge bug with All The Problems Anyone Ever Ran Into With DNF 3 will not be useful for reporters or maintainers, it'll just be a confusing mess, so trying to avoid that happening. Thanks.

Comment 4 Adam Williamson 2018-07-04 17:42:20 UTC
I'm not *sure* if both #c0 and #c1 have the same cause (they're both basically issues with database values that should be unique not being unique, but not the *same* values), but let's not split them up just yet, unless a DNF maintainer says to do so.

Comment 5 Mattias Ellert 2018-07-05 07:21:59 UTC
(In reply to Adam Williamson from comment #3)
> Mattias: those both look like different problems, so can you please file
> separate bugs for them?

Done, see bug 1598336.

Comment 6 Paul DeStefano 2018-07-07 21:02:31 UTC
Also, "Software" app hangs.  I think this the same root cause and is not a surprise, but should be mentioned in case someone else is looking for that symptom.

Comment 7 George R. Goffe 2018-07-09 17:03:42 UTC
Hi,

Since dnf is so screwed up I thought I'd try yum. I get the feeling that it would have worked except for one of the packages has a problem. I used this command "yum-deprecated upgrade dnf'*'" and got this for my trouble: Error: Invalid version flag: or

Apparently one of the packages has a problem.

George...

Comment 8 George R. Goffe 2018-07-09 21:35:05 UTC
This problem is not a package problem, it's a yum-deprecated problem... somewhere in the bowels of yum-deprecated. Can "we" get someone to fix this problem please?

Comment 9 Adam Williamson 2018-07-09 21:44:57 UTC
Paul "I think this the same root cause and is not a surprise"

it's quite unlikely. Please file it separately.

Comment 10 George R. Goffe 2018-07-10 05:29:15 UTC
Adam,

Was your last message meant for me?

George...

Comment 11 Adam Williamson 2018-07-10 05:47:31 UTC
No. That's why it says "Paul" at the start of it.

Comment 12 Paul DeStefano 2018-07-21 19:28:56 UTC
Any idea about a workaround for this?  I cannot upgrade now.

Comment 13 Andre Robatino 2018-07-21 19:31:26 UTC
See https://bugzilla.redhat.com/show_bug.cgi?id=1598590#c3 .

Comment 14 George R. Goffe 2018-07-22 05:31:32 UTC
I have renamed /var/lib/dnf/history and started an upgrade about 10 hours ago. It's still running (>7k upgrades). Dnf created a /var/lib/dnf/history.sqlite.

George...

Comment 15 Adam Williamson 2018-07-23 16:34:12 UTC
Could you attach the /var/lib/dnf/history to the bug, assuming you don't consider any of its likely contents (I'm assuming, all your previous DNF transactions) private? Thanks!

Comment 16 George R. Goffe 2018-07-23 18:28:38 UTC
Adam,

I would like to do this but the file (history-2017-06-27.sqlite) is rather large: 5,570,596,864 bytes or gzip'd (742712270).

Is there a site where I can download this file?

George...

Comment 17 Adam Williamson 2018-07-23 21:11:27 UTC
Yikes! I didn't realize it was that big. Perhaps a Google Drive share would work?

Comment 18 George R. Goffe 2018-07-23 23:21:54 UTC
Adam,

RedHat doesn't have a blind ftp server?

I'm not familiar with Google Drive.

George...

Comment 19 George R. Goffe 2018-07-23 23:46:35 UTC
Adam,

You should have email from me...

George...

Comment 20 George R. Goffe 2018-07-26 04:10:29 UTC
Adam,

I think I have a smaller history.sqlite file if you're interested.

George...

Comment 21 George R. Goffe 2018-08-02 20:10:46 UTC
Adam,

I'm seeing more of these problems. I think I'm seeing a correlation between multiple instances of dnf running where the failures occur. I'm seeing a message about the "DB being locked".

George...

Comment 22 Adam Williamson 2018-08-03 00:19:16 UTC
Thanks. I did get the file mail, btw, just haven't had time to check with dnf folks if it's of interest to them yet.

So, let me understand what you're experiencing here: you are seeing the same crashes again, and each time you have to wipe the history file to make DNF work again? Or what?

Comment 23 George R. Goffe 2018-08-03 04:31:21 UTC
Created attachment 1472876 [details]
tar gzip'd file containing console output from failure

Adam,

Essentially you are correct.

The database gets locked somehow and the attempt fails.

Here's my latest attempt that has failed although sometimes dnf succeeds.

George...

Comment 24 George R. Goffe 2018-08-03 04:32:38 UTC
Adam,

In the above case, dnf was running already. Argh!!!

George...

Comment 25 George R. Goffe 2018-08-11 17:00:23 UTC
Adam,

More info... Note that dnf was already running (see the end of this data):

dnf erase environment-modules
Traceback (most recent call last):
  File "/bin/dnf", line 58, in <module>
    main.user_main(sys.argv[1:], exit_code=True)
  File "/usr/lib/python3.7/site-packages/dnf/cli/main.py", line 179, in user_main
    errcode = main(args)
  File "/usr/lib/python3.7/site-packages/dnf/cli/main.py", line 64, in main
    return _main(base, args, cli_class, option_parser_class)
  File "/usr/lib/python3.7/site-packages/dnf/cli/main.py", line 99, in _main
    return cli_run(cli, base)
  File "/usr/lib/python3.7/site-packages/dnf/cli/main.py", line 123, in cli_run
    ret = resolving(cli, base)
  File "/usr/lib/python3.7/site-packages/dnf/cli/main.py", line 146, in resolving
    base.resolve(cli.demands.allow_erasing)
  File "/usr/lib/python3.7/site-packages/dnf/base.py", line 791, in resolve
    self.history)
  File "/usr/lib64/python3.7/site-packages/hawkey/__init__.py", line 220, in push_userinstalled
    user_installed = query.userinstalled(history.swdb)
  File "/usr/lib/python3.7/site-packages/dnf/db/history.py", line 290, in swdb
    self._swdb = libdnf.transaction.Swdb(self.dbpath)
  File "/usr/lib64/python3.7/site-packages/libdnf/transaction.py", line 713, in __init__
    this = _transaction.new_Swdb(*args)
RuntimeError: C++ std::exception: Exec failed: database is locked
Warning: ssh still initialized; probably ssh_init() was called more than once (init count: 1)
fc27-bash 4.4 /export/home/cfake# psg dnf
new n=/usr/bin/ps -efadlc | /usr/bin/grep -v grep | /usr/bin/egrep -i \(WCHAN\|dnf\)<...
.....
F S UID          PID    PPID CLS PRI ADDR SZ WCHAN  STIME TTY          TIME CMD
0 S root       14850       1 TS   19 - 184800 x64_sy 08:47 ?       00:00:01 /usr/bin/python3 /bin/dnfdragora-updater
4 R root       91694   17467 TS   19 - 187422 -     09:56 pts/16   00:00:11 /usr/bin/python3 /bin/dnf -y --best upgrade --allowerasing p11-kit-devel.x86_64 p11-kit-trust.x86_64 p11-kit.i686 p11-kit.x86_64

Comment 26 Jan Kurik 2018-08-14 10:25:37 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 29 development cycle.
Changing version to '29'.

Comment 27 Adam Williamson 2018-09-14 19:12:29 UTC
*** Bug 1598590 has been marked as a duplicate of this bug. ***

Comment 28 Adam Williamson 2018-09-14 19:13:13 UTC
Note the bug I just marked as dupe has a history file that the reporter says reliably triggers the bug:

https://bugzilla.redhat.com/attachment.cgi?id=1458556

Comment 29 Adam Williamson 2018-09-14 19:13:51 UTC
*** Bug 1624682 has been marked as a duplicate of this bug. ***

Comment 30 Geoffrey Marr 2018-09-17 20:12:49 UTC
Discussed during the 2018-09-17 blocker review meeting: [1]

The decision to classify this bug as an "AcceptedBlocker" was made as it violates the following criteria:

"The installed system must be able appropriately to install, remove, and update software with the default console tool for the relevant software type"

We will ask the dnf team for more investigation of this.

[1] https://meetbot.fedoraproject.org/fedora-blocker-review/2018-09-17/f29-blocker-review.2018-09-17-16.02.txt

Comment 31 Daniel Mach 2018-09-20 14:52:32 UTC
I believe I have a fix.
I'll write several unit tests and post a patch tomorrow.
It should be part of the next DNF build.

Comment 32 George R. Goffe 2018-09-20 15:15:46 UTC
Daniel,

It does seem that parts of dnf are not expecting a situation where there are two and I presume more dnf processes running and produce a stack dump.

I was just running a "dnf reinstall libreoffice'*'" and a "dnf upgrade liborcus" and got a stack dump. If a dnf resource needs to be serialized then dnf should wait on it's release, if not, dnf should run without taking a dump.

George...

Comment 33 Adam Williamson 2018-09-20 16:31:18 UTC
George: I think that may be a different problem from this. If you don't see the same traceback, it is likely not the same bug.

If you can reproduce the bug in #c25 reliably, it would be good if you could report that separately, as it should definitely be dealt with. Thanks.

Comment 34 Adam Williamson 2018-09-20 20:36:27 UTC
George: I think https://bugzilla.redhat.com/show_bug.cgi?id=1631533 should be the bug for any time you hit the "Exec failed: database is locked" traceback. That one is definitely not the same thing as the "UNIQUE constraint failed" crash.

Comment 35 Adam Williamson 2018-09-25 16:20:09 UTC
I believe https://github.com/rpm-software-management/libdnf/commit/d41f9a2f9879712cef8c5316d091d2e99fb1347a may be the intended fix for this, is that correct, dmach? It is in libdnf 0.20.0, which was recently tagged.

Comment 36 Adam Williamson 2018-09-30 16:30:53 UTC
*** Bug 1634305 has been marked as a duplicate of this bug. ***

Comment 37 Miroslav Suchý 2018-10-02 19:57:16 UTC
I can confirm that with libdnf 0.20.0 the traceback is gone.

Comment 38 Adam Williamson 2018-10-02 20:44:29 UTC
Let's call this POST, then, as I'm still waiting to hear from dmach what he wants to do about the DNF 3.6 / libdnf 0.20 update for F29.

Comment 39 Fedora Update System 2018-10-02 21:19:00 UTC
anaconda-29.24.3-2.fc29 dnf-3.6.1-1.fc29 dnf-plugins-core-3.0.4-1.fc29 dnf-plugins-extras-3.0.2-1.fc29 libdnf-0.20.0-1.fc29 lorax-29.12-3.fc29 has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2018-abc8825d92

Comment 40 Fedora Update System 2018-10-02 21:19:17 UTC
anaconda-29.24.3-2.fc29 dnf-3.6.1-1.fc29 dnf-plugins-core-3.0.4-1.fc29 dnf-plugins-extras-3.0.2-1.fc29 libdnf-0.20.0-1.fc29 lorax-29.12-3.fc29 has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2018-abc8825d92

Comment 41 Fedora Update System 2018-10-02 23:15:18 UTC
anaconda-29.24.3-2.fc29, dnf-3.6.1-1.fc29, dnf-plugins-core-3.0.4-1.fc29, dnf-plugins-extras-3.0.2-1.fc29, libdnf-0.20.0-1.fc29, lorax-29.12-3.fc29 has been pushed to the Fedora 29 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-abc8825d92

Comment 42 Fedora Update System 2018-10-07 20:59:23 UTC
anaconda-29.24.3-2.fc29, dnf-3.6.1-1.fc29, dnf-plugins-core-3.0.4-1.fc29, dnf-plugins-extras-3.0.2-1.fc29, libdnf-0.20.0-1.fc29, lorax-29.12-3.fc29 has been pushed to the Fedora 29 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.