Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1561209 - module does not properly track stream when another package Obsoletes: it
Summary: module does not properly track stream when another package Obsoletes: it
Keywords:
Status: NEW
Alias: None
Product: Fedora
Classification: Fedora
Component: distribution
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Martin Curlej
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: RejectedBlocker
Depends On:
Blocks: 1478068 1537253
TreeView+ depends on / blocked
 
Reported: 2018-03-27 21:40 UTC by Langdon White
Modified: 2020-10-31 21:55 UTC (History)
15 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-05-26 18:21:07 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Langdon White 2018-03-27 21:40:43 UTC
Description of problem:
django:1.6 is enabled however, python2-django is being "updated" by bare rpms.

replicated in 
* https://kojipkgs.fedoraproject.org/compose/28/Fedora-28-20180326.0/compose/Server/x86_64/iso/Fedora-Server-dvd-x86_64-28_Beta-1.1.iso
* docker.io/langdon/addon-modular-boltron:28

Version-Release number of selected component (if applicable):
dnf-0:2.7.5-8.fc28

How reproducible:
always

Log:
$ dnf enable django:1.6
Fedora Modular 28 - x86_64                      1.5 MB/s | 244 kB     00:00    
Fedora Modular 28 - x86_64 - Updates            2.7 kB/s | 257  B     00:00    
Fedora Modular 28 - x86_64 - Test Updates       1.4 MB/s | 129 kB     00:00    
Fedora 28 - x86_64 - Test Updates               8.9 MB/s |  12 MB     00:01    
Last metadata expiration check: 0:00:00 ago on Tue Mar 27 21:28:03 2018.
'django:1.6' is enabled

$ dnf install -y django
Last metadata expiration check: 0:01:02 ago on Tue Mar 27 21:28:03 2018.
Dependencies resolved.
================================================================================
 Package            Arch   Version                         Repository      Size
================================================================================
Installing:
 python2-django     noarch 1.6.11.7-1.module_1560+089ce146 fedora-modular 4.0 M
Installing dependencies:
 python-django-bash-completion
                    noarch 1.6.11.7-1.module_1560+089ce146 fedora-modular  21 k

Transaction Summary
================================================================================
Install  2 Packages

Total download size: 4.0 M
Installed size: 15 M
Downloading Packages:
(1/2): python-django-bash-completion-1.6.11.7-1  79 kB/s |  21 kB     00:00    
(2/2): python2-django-1.6.11.7-1.module_1560+08 5.8 MB/s | 4.0 MB     00:00    
--------------------------------------------------------------------------------
Total                                           4.5 MB/s | 4.0 MB     00:00     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                        1/1 
  Installing       : python-django-bash-completion-1.6.11.7-1.module_1560   1/2 
  Installing       : python2-django-1.6.11.7-1.module_1560+089ce146.noarc   2/2 
  Running scriptlet: python2-django-1.6.11.7-1.module_1560+089ce146.noarc   2/2 
  Verifying        : python2-django-1.6.11.7-1.module_1560+089ce146.noarc   1/2 
  Verifying        : python-django-bash-completion-1.6.11.7-1.module_1560   2/2 

Installed:
  python2-django.noarch 1.6.11.7-1.module_1560+089ce146                         
  python-django-bash-completion.noarch 1.6.11.7-1.module_1560+089ce146          

Complete!

dnf list installed "*django*"
Installed Packages
python-django-bash-completion.noarch 1.6.11.7-1.module_1560+089ce146 @fedora-modular
python2-django.noarch 1.6.11.7-1.module_1560+089ce146 @fedora-modular

$ dnf update -y "*django*"
Last metadata expiration check: 0:01:20 ago on Tue Mar 27 21:28:03 2018.
Dependencies resolved.
================================================================================
 Package                Arch       Version            Repository           Size
================================================================================
Installing dependencies:
 python2-django1.11     noarch     1.11.11-1.fc28     updates-testing     4.2 M
     replacing  python2-django.noarch 1.6.11.7-1.module_1560+089ce146

Transaction Summary
================================================================================
Install  1 Package

Total download size: 4.2 M
Installed size: 16 M
Downloading Packages:
python2-django1.11-1.11.11-1.fc28.noarch.rpm    322 kB/s | 4.2 MB     00:13    
--------------------------------------------------------------------------------
Total                                           318 kB/s | 4.2 MB     00:13     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                        1/1 
  Installing       : python2-django1.11-1.11.11-1.fc28.noarch               1/2 
  Obsoleting       : python2-django-1.6.11.7-1.module_1560+089ce146.noarc   2/2 
  Running scriptlet: python2-django-1.6.11.7-1.module_1560+089ce146.noarc   2/2 
  Verifying        : python2-django1.11-1.11.11-1.fc28.noarch               1/2 
  Verifying        : python2-django-1.6.11.7-1.module_1560+089ce146.noarc   2/2 

Installed:
  python2-django1.11.noarch 1.11.11-1.fc28                                      

Complete!

$ dnf list installed "*django*"
Installed Packages
python-django-bash-completion.noarch
                                1.6.11.7-1.module_1560+089ce146 @fedora-modular 
python2-django1.11.noarch       1.11.11-1.fc28                  @updates-testing

^^ python2-django version should not have changed

Comment 1 Stephen Gallagher 2018-03-27 22:15:59 UTC
OK, this is a special circumstance and one we didn't account for.

What happened here is that a new package was added to Fedora to be parallel-installable with python-django 2.x. This python2-django1.11 package added `Obsoletes: python2-django < 2` to itself, which is causing it to tell DNF "I should be used instead of any package calling itself python2-django with a version less than 2".

The bug here is that DNF's module handling doesn't handle the name change in this situation (it would be the same problem if we had just been going from python-django to python2-django). We probably need the depsolving logic to somehow figure out that because this package came from an enabled module, it's not eligible to be renamed unless the replacement is coming from another stream of the same module.

That all being said, I'd vote -1 to block on this for Beta and probably for Final because it really is an unlikely edge-case. The fact that it happened to hit one of our prototype modules is unfortunate, but it's mostly because the Django folks decided to carry a compat module instead of packaging it as a module stream.

Comment 2 Fedora Blocker Bugs Application 2018-03-27 22:20:07 UTC
Proposed as a Blocker for 28-beta by Fedora user sgallagh using the blocker tracking app because:

 FESCo identified a set of criteria that modularity was required to meet[1] in order for Fedora 28 Beta to ship. One of those requirements is that updating modules must use the available updates from the fedora-modular-updates[-testing] repos appropriately.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1537253#c6

Comment 3 Stephen Gallagher 2018-03-27 22:22:57 UTC
I proposed this as a blocker because I think it needs the review of more people, but as I noted above I'm -1 to block Beta on this. The functionality works in most cases, which is the spirit of the criterion. This is an edge case, and certainly a bug we need to fix, but I don't know that we have to fix it *for Beta*. I'd entertain an argument to block Final on it though.

Comment 4 Adam Williamson 2018-03-28 01:28:43 UTC
With Stephen's explanation, I think I'm -1 for Beta at least.

Comment 5 Kevin Fenzi 2018-03-28 01:38:48 UTC
-1 beta blocker.

Comment 6 Mohan Boddu 2018-03-28 02:10:12 UTC
-1 Beta blocker but definitely something to look for F28 Final.

Comment 7 sumantro 2018-03-28 03:40:09 UTC
-1 on beta blocker/+1 on F28 Final

Comment 8 Petr Šabata 2018-03-28 08:01:54 UTC
Oh, this is bad.  But +1 to the blocker suggestions.

Comment 9 Petr Šabata 2018-03-28 08:07:39 UTC
This can be reproduced with the latest dnf, 2.7.5-9.fc28.

Comment 10 Daniel Mach 2018-03-28 11:14:06 UTC
The behavior is expected and according to current specifications.
If DNF should behave differently, could anyone put together specs for the expected behavior?

Comment 11 Stephen Gallagher 2018-03-28 12:01:17 UTC
(In reply to Daniel Mach from comment #10)
> The behavior is expected and according to current specifications.
> If DNF should behave differently, could anyone put together specs for the
> expected behavior?

I'm not sure I'd agree that it's expected (at least from a user perspective). There's an argument to be made here that there's a bug in our packaging policy that allowed this situation to occur, of course.

I think the additional specification is that Obsoletes: should be ignored that would cause a package in an enabled module to be superseded by a package from anywhere except another stream of that same module.

Now... poke holes in that until we find the right phrasing :)

Comment 12 Matthew Miller 2018-03-28 14:54:53 UTC
-1 blocker for beta, but I'd like this resolved for final (either by changes to dnf or to packaging policy -- don't care).

Comment 13 Stephen Gallagher 2018-03-28 16:36:35 UTC
For this *specific* module, I'm going to implement a workaround and just have the modular version explicitly Obsoletes the non-modular one. But we'll leave this ticket open for DNF to come up with a correct implementation down the road.

Comment 14 Fedora Update System 2018-03-28 18:01:59 UTC
django-1.6-20180328170906.c2c572ec has been submitted as an update to Fedora 28 Modular. https://bodhi.fedoraproject.org/updates/FEDORA-MODULAR-2018-0e193b5c1a

Comment 15 Fedora Update System 2018-03-28 21:31:51 UTC
django-1.6-20180328170906.c2c572ec has been pushed to the Fedora 28 Modular testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-MODULAR-2018-0e193b5c1a

Comment 16 Stephen Gallagher 2018-03-29 01:45:39 UTC
I tested this in a container that was pointed at a synchronized updates-testing repo tonight. The workaround appears to behave correctly for the Django module (it no longer offers to upgrade the 1.6 version from the module to the 1.11 version from the base repo).

Comment 17 Neal Gompa 2018-03-29 18:16:12 UTC
(In reply to Daniel Mach from comment #10)
> The behavior is expected and according to current specifications.
> If DNF should behave differently, could anyone put together specs for the
> expected behavior?

Daniel, it sounds like we need a sticky vendor implementation in DNF to resolve this issue.

libsolv has the ability to filter updates based on "vendor", which is traditionally defined by repo ID. In this case, we'd probably want a second attribute from the repodata that can be used to indicate that it's the same "vendor", since we're not splitting modules into their own repositories.

Having a sticky vendor implementation would generally be useful for people who use third party repositories or COPRs to replace Fedora packages, too, so they don't get surprised with them getting switched randomly.

Comment 18 Adam Williamson 2018-03-29 20:08:43 UTC
Discussed at 2018-03-29 go/no-go meeting, acting as a blocker review meeting: https://meetbot-raw.fedoraproject.org/fedora-meeting/2018-03-29/f28-beta-go-no-go-meeting.2018-03-29-16.04.html . Rejected as a Beta blocker on the grounds this is quite a corner case, and the known instance of it has been worked around for Beta: we don't think it's enough of a violation of the requirements in https://bugzilla.redhat.com/show_bug.cgi?id=1537253#c6 to constitute a blocker.

Comment 19 Fedora Update System 2018-04-26 20:46:46 UTC
django-1.6-20180328170906.c2c572ec has been pushed to the Fedora 28 Modular stable repository. If problems still persist, please make note of it in this bug report.

Comment 20 Ben Cotton 2019-05-02 20:47:41 UTC
This message is a reminder that Fedora 28 is nearing its end of life.
On 2019-May-28 Fedora will stop maintaining and issuing updates for
Fedora 28. It is Fedora's policy to close all bug reports from releases
that are no longer maintained. At that time this bug will be closed as
EOL if it remains open with a Fedora 'version' of '28'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 28 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 21 Adam Williamson 2019-05-21 21:35:16 UTC
Stephen, would you say we still need DNF folks to look at doing something here?

Comment 22 Ben Cotton 2019-05-28 19:01:11 UTC
Fedora 28 changed to end-of-life (EOL) status on 2019-05-28. Fedora 28 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 23 Stephen Gallagher 2019-06-03 12:28:10 UTC
Sorry, I missed this in the flood of EOL notifications.

So, I do think that we should have some protection against this. We have a stated requirement that DNF should never attempt to upgrade from a modular package to a non-modular one (specifically for the case where the module metadata gets misplaced) and I think that basically applies here too.

Comment 24 Jaroslav Mracek 2019-09-28 17:23:19 UTC
DNF-4 modular filtering is based on the name search and on the provide search. In case of python2-django1.11 the issue would be resolved if python2-django1.11 will provide python2-django.

In case that we want to restrict upgrade using vendor, the approach must be verified by Modularity team and distribution itself => changing component.

Comment 25 Ben Cotton 2020-04-30 20:24:46 UTC
This message is a reminder that Fedora 30 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 30 on 2020-05-26.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '30'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 30 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 26 Ben Cotton 2020-05-26 18:21:07 UTC
Fedora 30 changed to end-of-life (EOL) status on 2020-05-26. Fedora 30 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.