Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1935465 - Cached rpms in dnf uses historic yum checksum code instead of librepo's checksum
Summary: Cached rpms in dnf uses historic yum checksum code instead of librepo's checksum
Keywords:
Status: ASSIGNED
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
low
Target Milestone: ---
Assignee: Matthew Almond
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: dnf-community
TreeView+ depends on / blocked
 
Reported: 2021-03-04 23:59 UTC by Matthew Almond
Modified: 2021-03-18 18:10 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Matthew Almond 2021-03-04 23:59:58 UTC
Description of problem: dnf has two implementations of rpm checksumming. One stemming from yum written in Python, embedded in dnf. Another in librepo which is authoritative for the normal download+install+delete flow.

Version-Release number of selected component (if applicable):

How reproducible: every time

# Direct install:

Steps to Reproduce:

1. dnf install bash --installroot=/var/tmp/direct.root --releasever=33


Expected/Actual results:

dnf downloads packages, verifies using librepo's lr_checksum_compare, and when these match, installs them.


# Cached install:
Steps to Reproduce:

1. dnf install bash --installroot=/var/tmp/cached.root  --releasever=33 --downloadonly
2. dnf install bash --installroot=/var/tmp/cached.root  --releasever=33 --cacheonly

Expected results:

Step 1 will download and verify the files, adding xattr to cache checksum
Step 2 will read checksum from xattr if present, falling back to lr_checksum_compare.

Actual results:

dnf uses dnf.yum.misc.checksum() in Package.verifyLocalPkg() which doesn't use the xattr cache. This works, but is slower, and is broken with CoW (see next bit)

Additional info:
Today this is a slight perf regression, and doesn't cause any visible failures. The long term issue is that there are two pieces of code that verify checksums and they're different. In the context of https://github.com/rpm-software-management/librepo/pull/222 this means failure because the updated code in librepo isn't being used in the lesss common path.

Comment 1 Matthew Almond 2021-03-05 00:03:12 UTC
I'm motivated to fix this. My initial thoughts are to modify librepo's python bindings, exposing lr_checksum_fd_compare + LrChecksumType from librepo/checksum.c. Then I can modify dnf/package.py to use that method directly, and delete the checksum code out of dnf/yum/misc.py

Comment 2 Matthew Almond 2021-03-11 23:19:37 UTC
Main fix:

https://github.com/rpm-software-management/libdnf/pull/1164
https://github.com/rpm-software-management/dnf/pull/1743

This replaces the old code with the librepo stuff properly.

A minor bug:
https://github.com/rpm-software-management/librepo/pull/234


Note You need to log in before you can comment on or make changes to this bug.