Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.

Bug 1925717

Summary: Upgrade removes all overrides
Product: [Fedora] Fedora Reporter: Andrew Thurman <ajtbecool>
Component: rpm-ostreeAssignee: Colin Walters <walters>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 33CC: dustymabe, jonathan, kxra, lucab, miabbott, pbrobinson, philip.wyett, robertthomasfairley, sdodson, suhanc, travier, walters, yuan
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
URL: https://bodhi.fedoraproject.org/updates/FEDORA-2021-937b45bf55
Whiteboard:
Fixed In Version: rpm-ostree-2021.1-4.fc33 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-02-11 01:43:15 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1269538    

Description Andrew Thurman 2021-02-05 23:33:27 UTC
User-Agent:       Mozilla/5.0 (X11; Linux x86_64; rv:85.0) Gecko/20100101 Firefox/85.0
Build Identifier: 

With `libsolv` > libsolv-0.7.15-1.fc33.x86_64.rpm `rpm-ostree upgrade` removes all overrides made by the user:

```sh
[andythurman@rockhopper ~]$ rpm-ostree status 
State: idle
Deployments:
● ostree://fedora:fedora/33/x86_64/silverblue
                   Version: 33.20210204.0 (2021-02-04T01:41:19Z)
                BaseCommit: 39ac11c939acaa8cc7bb53635bd946b7c76ac74a805272ed1720b615b75dfcb4
              GPGSignature: Valid signature by 963A2BEB02009608FE67EA4249FD77499570FF31
       RemovedBasePackages: gnome-software gnome-software-rpm-ostree 3.38.0-2.fc33, firefox 85.0-8.fc33
           LayeredPackages: abrt-desktop fedora-workstation-repositories totem
[andythurman@rockhopper ~]$ rpm-ostree upgrade 
2 metadata, 0 content objects fetched; 788 B transferred in 1 seconds; 0 bytes content written
Checking out tree 635d8c6... done
Inactive base removals:
  gnome-software
  gnome-software-rpm-ostree
  firefox
Enabled rpm-md repositories: fedora-cisco-openh264 updates fedora updates-archive
Updating metadata for 'fedora-cisco-openh264'... done
rpm-md repo 'fedora-cisco-openh264'; generated: 2020-08-25T19:10:34Z
Updating metadata for 'updates'... done
rpm-md repo 'updates'; generated: 2021-02-05T01:01:46Z
Updating metadata for 'fedora'... done
rpm-md repo 'fedora'; generated: 2020-10-19T23:27:19Z
Updating metadata for 'updates-archive'... done
rpm-md repo 'updates-archive'; generated: 2021-02-05T06:27:48Z
Importing rpm-md... done
Resolving dependencies... done
Will download: 86 packages (22.5 MB)
Downloading from 'updates'... done
Downloading from 'fedora'... done
Downloading from 'fedora-cisco-openh264'... done
Importing packages... done
Checking out packages... done
Running pre scripts... done
Running post scripts... done
Running posttrans scripts... done
Writing rpmdb... done
error: Sanity-checking final rpmdb: Didn't find package 'gstreamer1-plugin-openh264-1.16.2-2.fc33.x86_64'
```

This issue is present in all versions, including `rpm-ostree-2021.1-3.fc33` (see bodhi link)

Reproducible: Always

Steps to Reproduce:
1. Run `rpm-ostree upgrade`
Actual Results:  
Overrided packages are removed. System cannot upgrade on versions less than `rpm-ostree-2021.1-3.fc33`

Expected Results:  
System upgrades smoothly

I have discovered a temporary, hacky fix:

```sh
cd Downloads
wget https://kojipkgs.fedoraproject.org//packages/rpm-ostree/2021.1/3.fc33/x86_64/rpm-ostree-2021.1-3.fc33.x86_64.rpm && wget https://kojipkgs.fedoraproject.org//packages/rpm-ostree/2021.1/3.fc33/x86_64/rpm-ostree-libs-2021.1-3.fc33.x86_64.rpm && wget https://kojipkgs.fedoraproject.org//packages/libsolv/0.7.15/1.fc33/x86_64/libsolv-0.7.15-1.fc33.x86_64.rpm
sudo rpm-ostree usroverlay && sudo rpm -Uvh rpm-ostree-{,libs-}*.rpm
systemctl restart rpm-ostreed.service
rpm-ostree override replace libsolv-0.7.15-1.fc33.x86_64.rpm
```

Comment 1 Colin Walters 2021-02-06 20:04:12 UTC
xref https://github.com/coreos/rpm-ostree/issues/2548

Comment 2 Colin Walters 2021-02-06 20:26:05 UTC
https://pagure.io/fedora-infrastructure/issue/9634

Comment 4 Fedora Update System 2021-02-10 17:33:44 UTC
FEDORA-2021-9091468793 has been submitted as an update to Fedora 33. https://bodhi.fedoraproject.org/updates/FEDORA-2021-9091468793

Comment 5 Fedora Update System 2021-02-11 01:43:15 UTC
FEDORA-2021-9091468793 has been pushed to the Fedora 33 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 6 Junjie Yuan 2021-02-11 16:12:21 UTC
If you are using Silverblue, you can rollback to 33.20210203.0 (Base Commit: 8776ab1b7ba55c1455c38f76ae4f050a26e6489029e91f76544e551998001667, libsolv 0.7.15-1.fc33), reboot and upgrade to apply the patch.