Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1083226 - spacewalk-repo-sync --exclude is excluding more than what's in the exclude list
Summary: spacewalk-repo-sync --exclude is excluding more than what's in the exclude list
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Spacewalk
Classification: Community
Component: Clients
Version: 2.2
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
Assignee: Stephen Herr
QA Contact: Red Hat Satellite QA List
URL:
Whiteboard:
Depends On: 1008802 1087669
Blocks: space22
TreeView+ depends on / blocked
 
Reported: 2014-04-01 17:41 UTC by Stephen Herr
Modified: 2014-07-17 08:40 UTC (History)
5 users (show)

Fixed In Version: spacewalk-backend-2.2.15-1
Doc Type: Bug Fix
Doc Text:
Clone Of: 1008802
Environment:
Last Closed: 2014-07-17 08:40:47 UTC
Embargoed:


Attachments (Terms of Use)

Comment 1 Stephen Herr 2014-04-01 17:43:38 UTC
Description of problem:
[root@rhel6sat55 ~]# spacewalk-repo-sync --channel bz965906 -u http://downloads.linux.hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/
Repo URL: http://downloads.linux.hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/
Packages in repo:               313
Packages already synced:        206
Packages to sync:                45
1/45 : kmod-hpahcisr-xen-rhel5-1.2.6-15.rhel5u7-0.x86_64
2/45 : kmod-hpahcisr-rhel5-1.2.6-15.rhel5u7-0.x86_64
3/45 : kmod-hpvsa-rhel5-1.2.2-3-0.x86_64
4/45 : kmod-mpt2sas-xen-15.10.01.00-1-0.x86_64
5/45 : hpmouse-1.2.1-1-0.noarch
6/45 : kmod-hpahcisr-rhel5-1.2.6-15.rhel5u8-0.x86_64
7/45 : kmod-hpahcisr-rhel5-1.2.6-11-0.x86_64
8/45 : kmod-cciss-rhel5-3.6.28-20-0.x86_64
9/45 : kmod-cciss-rhel5-3.6.28-22.rhel5u9-0.x86_64
10/45 : kmod-cciss-xen-rhel5-3.6.28-22.rhel5u9-0.x86_64
11/45 : kmod-hpahcisr-xen-rhel5-1.2.6-11-0.x86_64
12/45 : kmod-be2iscsi-xen-4.6.247.3-1-0.x86_64
13/45 : kmod-cciss-rhel5-3.6.28-20-0.x86_64
14/45 : kmod-cciss-rhel5-3.6.28-22.rhel5u8-0.x86_64
15/45 : kmod-cciss-rhel5-3.6.28-17-0.x86_64
16/45 : kmod-mpt2sas-15.10.01.00-1-0.x86_64
17/45 : kmod-mpt2sas-xen-15.10.01.00-1-0.x86_64
18/45 : kmod-be2net-xen-4.6.247.7-1-0.x86_64
19/45 : kmod-cciss-rhel5-3.6.28-22.rhel5u7-0.x86_64
20/45 : kmod-mpt2sas-15.10.01.00-1-0.x86_64
21/45 : kmod-mpt2sas-13.10.02.00-1-0.x86_64
22/45 : kmod-mpt2sas-xen-15.10.01.00-1-0.x86_64
23/45 : kmod-hpahcisr-rhel5-1.2.6-15.rhel5u9-0.x86_64
24/45 : kmod-mpt2sas-15.10.01.00-1-0.x86_64
25/45 : kmod-cciss-xen-rhel5-3.6.28-17-0.x86_64
26/45 : kmod-hpahcisr-xen-rhel5-1.2.6-15.rhel5u8-0.x86_64
27/45 : kmod-be2net-4.6.247.7-1-0.x86_64
28/45 : kmod-cciss-xen-rhel5-3.6.28-22.rhel5u8-0.x86_64
29/45 : kmod-cciss-xen-rhel5-3.6.28-17-0.x86_64
30/45 : kmod-elx-lpfc-xen-8.2.2.21.2p-1-0.x86_64
31/45 : kmod-cciss-xen-rhel5-3.6.28-20-0.x86_64
32/45 : kmod-hpahcisr-rhel5-1.2.6-13-0.x86_64
33/45 : kmod-mpt2sas-xen-13.10.02.00-2-0.x86_64
34/45 : kmod-mpt2sas-xen-13.10.02.00-1-0.x86_64
35/45 : kmod-cciss-xen-rhel5-3.6.28-22.rhel5u7-0.x86_64
36/45 : kmod-hpahcisr-xen-rhel5-1.2.6-15.rhel5u9-0.x86_64
37/45 : kmod-elx-lpfc-8.2.2.21.2p-1-0.x86_64
38/45 : hp-ilo-8.5.0-1.rhel5-0.x86_64
39/45 : kmod-be2iscsi-4.6.247.3-1-0.x86_64
40/45 : kmod-mpt2sas-xen-10.10.22.00-1-0.x86_64
41/45 : kmod-cciss-xen-rhel5-3.6.28-20-0.x86_64
42/45 : kmod-hpvsa-xen-rhel5-1.2.2-3-0.x86_64
43/45 : kmod-cciss-rhel5-3.6.28-17-0.x86_64
44/45 : hp-OpenIPMI-8.5.2-85.rhel5-0.x86_64
45/45 : kmod-mpt2sas-13.10.02.00-2-0.x86_64
Repo http://downloads.linux.hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/ has 0 errata.
Sync completed.
Total time: 0:01:02
[root@rhel6sat55 ~]# spacewalk-repo-sync --channel bz965906 -u http://downloads.linux.hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/ -e hpmouse
Repo URL: http://downloads.linux.hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/
Packages in repo:               313
Packages passed filter rules:   220
Packages already synced:        206
Packages to sync:                14
1/14 : hp-OpenIPMI-8.5.2-85.rhel5-0.x86_64
2/14 : kmod-cciss-rhel5-3.6.28-22.rhel5u8-0.x86_64
3/14 : kmod-hpahcisr-rhel5-1.2.6-15.rhel5u8-0.x86_64
4/14 : kmod-cciss-xen-rhel5-3.6.28-22.rhel5u9-0.x86_64
5/14 : hp-ilo-8.5.0-1.rhel5-0.x86_64
6/14 : kmod-hpahcisr-rhel5-1.2.6-15.rhel5u7-0.x86_64
7/14 : kmod-hpahcisr-rhel5-1.2.6-15.rhel5u9-0.x86_64
8/14 : kmod-cciss-rhel5-3.6.28-22.rhel5u9-0.x86_64
9/14 : kmod-cciss-xen-rhel5-3.6.28-22.rhel5u7-0.x86_64
10/14 : kmod-hpahcisr-xen-rhel5-1.2.6-15.rhel5u9-0.x86_64
11/14 : kmod-cciss-rhel5-3.6.28-22.rhel5u7-0.x86_64
12/14 : kmod-cciss-xen-rhel5-3.6.28-22.rhel5u8-0.x86_64
13/14 : kmod-hpahcisr-xen-rhel5-1.2.6-15.rhel5u8-0.x86_64
14/14 : kmod-hpahcisr-xen-rhel5-1.2.6-15.rhel5u7-0.x86_64
Repo http://downloads.linux.hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/ has 0 errata.
Sync completed.
Total time: 0:00:22
[root@rhel6sat55 ~]# 

I am just excluding a single package - hpmouse - but the number of packages to sync goes from 45 to 14 when I specify -e hpmouse.  It should go from 45 to 44.

Version-Release number of selected component (if applicable):
spacewalk-backend-tools-1.7.38-47.el6sat.noarch

How reproducible:
Always

Steps to Reproduce:
1. Get packages to sync: spacewalk-repo-sync --channel bz965906 -u http://downloads.linux.hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/
2. Try to sync just the hpmouse package: spacewalk-repo-sync --channel bz965906 -u http://downloads.linux.hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/ -e hpmouse
3. Instead of 1 package being excluded, many are excluded.

Actual results:
Too many packages are excluded.

Expected results:
Just those packages specified in --exclude list should be excluded ... shouldn't they?

Additional info:
This bug was noticed when troubleshooting https://bugzilla.redhat.com/show_bug.cgi?id=965906.  This is preventing the customer from using the --exclude option as a workaround for the problem they are encountering in the BZ.

--- Additional comment from Mark Huth on 2013-09-17 02:32:04 EDT ---

The problem could be in how the package dependencies are computed.  When I commented out line 154 in /usr/lib/python2.6/site-packages/spacewalk/satellite_tools/repo_plugins/yum_src.py, the --exclude option seems to work as expected (ie, only a single package was excluded in the final list):

147     def list_packages(self, filters):
148         """ list packages"""
149         self.sack.populate(self.repo, 'metadata', None, 0)
150         pkglist = self.sack.returnPackages()
151         self.num_packages = len(pkglist)
152         if filters:
153             pkglist = self._filter_packages(pkglist, filters)
154             #pkglist = self._get_package_dependencies(self.sack, pkglist)
155             self.num_excluded = self.num_packages - len(pkglist)

[root@rhel6sat55 ~]# spacewalk-repo-sync --channel bz965906 -u http://downloads.linux.hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/ -e hpmouse
Repo URL: http://downloads.linux.hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/
Packages in repo:               313
Packages passed filter rules:   250
Packages already synced:        206
Packages to sync:                44
1/44 : kmod-hpahcisr-xen-rhel5-1.2.6-15.rhel5u7-0.x86_64
...
44/44 : kmod-mpt2sas-13.10.02.00-2-0.x86_64
Repo http://downloads.linux.hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/ has 0 errata.
Sync completed.
Total time: 0:01:20

--- Additional comment from Mark Huth on 2013-09-18 01:46:23 EDT ---

Seems we could still be being bitten by the problem with unique packages in HP's repo:

Looking at these lines in /usr/lib/python2.6/site-packages/spacewalk/satellite_tools/repo_plugins/yum_src.py:

        if filters:
            pkglist = self._filter_packages(pkglist, filters)
            pkglist = self._get_package_dependencies(self.sack, pkglist)

Seems there are duplicate packages in the list:

[root@rhel6sat55 repo_plugins]# spacewalk-repo-sync --channel bz965906 -u http://downloads.linux.hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/ -e hpmouse
> /usr/lib/python2.6/site-packages/spacewalk/satellite_tools/repo_plugins/yum_src.py(154)list_packages()
-> if filters:
(Pdb) n
> /usr/lib/python2.6/site-packages/spacewalk/satellite_tools/repo_plugins/yum_src.py(155)list_packages()
-> pkglist = self._filter_packages(pkglist, filters)
(Pdb) 
> /usr/lib/python2.6/site-packages/spacewalk/satellite_tools/repo_plugins/yum_src.py(156)list_packages()
-> pkglist = self._get_package_dependencies(self.sack, pkglist)
(Pdb) p1 = pkglist[:]
(Pdb) len(p1)
312              <--- there are 312 packages in the list after the _filter_packages() method
(Pdb) n
> /usr/lib/python2.6/site-packages/spacewalk/satellite_tools/repo_plugins/yum_src.py(157)list_packages()
-> self.num_excluded = self.num_packages - len(pkglist)
(Pdb) p2 = pkglist[:]
(Pdb) len(p2)
282             <--- only 282 packages after the _get_package_dependencies() method


But it seems the p1 package list has duplicate packages because if we turn p1 into a set then the number of unique set elements in 282:

(Pdb) from sets import Set
/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/repo_plugins/yum_src.py:1: DeprecationWarning: the sets module is deprecated
  #
(Pdb) s1 = Set(p1)
(Pdb) s2 = Set(p2)
(Pdb) len(s1)
282
(Pdb) len(s2)
282


And its the yum.misc.unique() method in _get_package_dependencies() that is removing the duplicate packages:

(Pdb) len(yum.misc.unique(p1))
282


So perhaps the --exclude option is working ok afterall and the problem still lies with the HP repo.  But is there *any* way this repo can be synced into Satellite without the problematic packages being synced every time?  Does spacewalk-repo-sync need a --skip-broken-repo option or something so it ignores these problematic packages?

--- Additional comment from Michael Mráka on 2013-10-14 05:35:19 EDT ---

There are two different issues.

a) Multiple versions of the same package in the repo causes re-download of them in subsequent spacewalk-repo-sync runs. Number of re-downloaded packages varies according to which version was downloaded in the previous run:

Sync started: Mon Oct 14 04:45:38 2013
['/usr/bin/spacewalk-repo-sync', '--channel', 'bz965906', '-u', 'http://downloads.linux.hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/']
Repo URL: http://downloads.linux.hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/
Packages in repo:               313
Packages already synced:          0
Packages to sync:               251
1/251 : cpqacuxe-9.20-9.0-0.i386
...
251/251 : kmod-mpt2sas-13.10.02.00-2-0.x86_64
Linking packages to channel.
Repo http://downloads.linux.hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/ has 0 errata.
Sync completed.
Total time: 0:12:32

Sync started: Mon Oct 14 05:00:04 2013
['/usr/bin/spacewalk-repo-sync', '--channel', 'bz965906', '-u', 'http://downloads.linux.hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/']
Repo URL: http://downloads.linux.hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/
Packages in repo:               313
Packages already synced:        221
Packages to sync:                30
1/30 : kmod-hpvsa-rhel5-1.2.2-3-0.x86_64
...
Sync completed.
Total time: 0:00:39

Sync started: Mon Oct 14 05:00:56 2013
['/usr/bin/spacewalk-repo-sync', '--channel', 'bz965906', '-u', 'http://downloads.linux.hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/']
Repo URL: http://downloads.linux.hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/
Packages in repo:               313
Packages already synced:        199
Packages to sync:                52
1/52 : kmod-be2iscsi-4.6.247.3-1-0.x86_64
...
Sync completed.
Total time: 0:00:52

Sync started: Mon Oct 14 05:02:02 2013
['/usr/bin/spacewalk-repo-sync', '--channel', 'bz965906', '-u', 'http://downloads.linux.hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/']
Repo URL: http://downloads.linux.hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/
Packages in repo:               313
Packages already synced:        221
Packages to sync:                30
1/30 : kmod-hpvsa-rhel5-1.2.2-3-0.x86_64
...
Sync completed.
Total time: 0:00:30

Sync started: Mon Oct 14 05:03:41 2013
['/usr/bin/spacewalk-repo-sync', '--channel', 'bz965906', '-u', 'http://downloads.linux.hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/']
Repo URL: http://downloads.linux.hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/
Packages in repo:               313
Packages already synced:        199
Packages to sync:                52
1/52 : kmod-be2iscsi-4.6.247.3-1-0.x86_64
...
Sync completed.
Total time: 0:00:51


Notice than number of packages to sync jumps from 30 to 52 to 30 to 52 ...

--- Additional comment from Michael Mráka on 2013-10-14 05:55:50 EDT ---

b) the second issue - filtering:

As a part of filtering feature spacewalk-repo-sync computes dependencies and include them to the list of to-be-downloaded packages. As a part of it yum subroutines makes the list of packages unique. Without exclude/include filter there's no package unification and list can contain duplicates.

If I exclude random non-existent package just to force spacewalk-repo-sync to make the list unique it will become consistent and stop re-downloading packages in each run.

Sync started: Mon Oct 14 05:51:39 2013
['/usr/bin/spacewalk-repo-sync', '--channel', 'bz965906', '-u', 'http://downloads.linux.
hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/', '--exclude', 'dups']
Repo URL: http://downloads.linux.hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/
Packages in repo:               313
Packages passed filter rules:   221
Packages already synced:        199
Packages to sync:                22
1/22 : kmod-cciss-rhel5-3.6.28-20-0.x86_64
...
Sync completed.
Total time: 0:00:19

Sync started: Mon Oct 14 05:52:09 2013
['/usr/bin/spacewalk-repo-sync', '--channel', 'bz965906', '-u', 'http://downloads.linux.hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/', '--exclude', 'dups']
Repo URL: http://downloads.linux.hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/
Packages in repo:               313
Packages passed filter rules:   221
No new packages to sync.
Repo http://downloads.linux.hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/ has 0 errata.
Sync completed.
Total time: 0:00:08

Sync started: Mon Oct 14 05:54:05 2013
['/usr/bin/spacewalk-repo-sync', '--channel', 'bz965906', '-u', 'http://downloads.linux.hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/', '--exclude', 'dups']
Repo URL: http://downloads.linux.hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/
Packages in repo:               313
Packages passed filter rules:   221
No new packages to sync.
Repo http://downloads.linux.hp.com/SDR/downloads/SPP/RHEL/5/x86_64/current/ has 0 errata.
Sync completed.
Total time: 0:00:05

--- Additional comment from Stephen Herr on 2014-04-01 11:16:41 EDT ---

(In reply to Michael Mráka from comment #5)
> There are two different issues.
> 
> a) Multiple versions of the same package in the repo causes re-download of
> them in subsequent spacewalk-repo-sync runs. Number of re-downloaded
> packages varies according to which version was downloaded in the previous
> run:

More specifically the root problem is that the yum repodata at the source is being generated incorrectly. The repository will contain say the following rpms:

kmod-mpt2sas-13.10.02.00-2.rhel5u7.x86_64.rpm
kmod-mpt2sas-13.10.02.00-2.rhel5u8.x86_64.rpm

They are both incorrectly getting written into the yum metadata like so:

name: kmod-mpt2sas
version: 13.10.02.00
release: 2
arch: x86_64

and with two different checksums. In *correct* yum metadata the rpms would have two different releases, "2.rhel5u7" and "2.rhel5u8", and yum would be then be able to tell them apart and everything would work well.

Comment 2 Stephen Herr 2014-04-01 17:50:29 UTC
I have opened a forum thread to alert HP to the problem, if anyone knows of a better place to file it let me know.
http://h30499.www3.hp.com/t5/Community-Feedback-Suggestions/Yum-metadata-being-generated-incorrectly-in-HP-yum-repos/td-p/6433730

The best solution that we have on the downstream side is to simply always uniquify the package list so we will get consistent behavior. The only real fix is to get HP to fix their broken metadata.

Committing to Spacewalk master:
0cbacb078efba7ec0b17eb9ff2eddd2efb0e1b4f

Comment 3 Milan Zázrivec 2014-07-17 08:40:47 UTC
Spacewalk 2.2 has been released:

    https://fedorahosted.org/spacewalk/wiki/ReleaseNotes22


Note You need to log in before you can comment on or make changes to this bug.