Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1936875 - Python generators for extras: Case sensitivity regression (e.g. in python-dns)
Summary: Python generators for extras: Case sensitivity regression (e.g. in python-dns)
Keywords:
Status: ON_QA
Alias: None
Product: Fedora
Classification: Fedora
Component: python-rpm-generators
Version: 34
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Tomas Orsava
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On: 1937944
Blocks: 1935835
TreeView+ depends on / blocked
 
Reported: 2021-03-09 11:51 UTC by Miro Hrončok
Modified: 2021-03-19 20:12 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Type: Bug
Embargoed:

Attachments (Terms of Use)

Description Miro Hrončok 2021-03-09 11:51:48 UTC 
It seems that the importlib.metadata change introduced a regression wrt case sensitivity in extras names:

python-dns has:

  %{?python_extras_subpkg:%python_extras_subpkg -n python3-dns -i %{python3_sitelib}/*.egg-info dnssec trio doh idna}

And the actual build with generators version 11 had:

  Provides: python-dns+dnssec = 2.1.0-2.fc34 python3-dns+dnssec = 2.1.0-2.fc34 python3.9-dns+dnssec = 2.1.0-2.fc34 python3.9dist(dnspython[dnssec]) = 2.1 python3dist(dnspython[dnssec]) = 2.1
  ...


While currently, it fails with:

  Error: The package name contains an extras name `dnssec` that was not found in the metadata.


See https://koschei.fedoraproject.org/package/python-dns?collection=f35



I've checked the metadata and it contains:

  [DNSSEC]
  cryptography>=2.6

So this is a case sensitivity issue. I am completely fine if we have to change the definition to use DNSSEC over dnssec, but currently the package name and provide is upper case:

  Provides: python-dns+DNSSEC = 2.1.0-3.fc35 python3-dns+DNSSEC = 2.1.0-3.fc35 python3.9-dns+DNSSEC = 2.1.0-3.fc35 python3.9dist(dnspython[DNSSEC]) = 2.1 python3dist(dnspython[DNSSEC]) = 2.1

That means:

 - nothing obsoletes the old python-dns+dnssec package
 - the %py3_dist macro cannot be used: %{py3_dist dnspython[DNSSEC]} -> python3dist(dnspython[dnssec])


I think we need to canonize the extras name before we put it in the provide (in Python) and we need to canonize it before we construct the package name (in Lua, but we have a macro).
Comment 1 Miro Hrončok 2021-03-09 12:10:57 UTC 
See also https://github.com/pypa/setuptools/issues/1608
Comment 2 Miro Hrončok 2021-03-09 12:30:05 UTC 
Good news: pip seem to treat dnspython[DNSSEC] and dnspython[dnssec] equally.

bad news: it does not treat webscrapbook[adhoc-ssl] same as webscrapbook[adhoc_ssl], so we cannot use the same canonization rules as we do for names.

I suspect we need to just lowercase it all, but we also need to to fix %py3_dist :(

Either way, we should rally check what rules are there and if none, define them upstream.
Comment 3 Miro Hrončok 2021-03-09 14:10:21 UTC 
Action plan:

Quick fix (we assume extras name are exchangeable only if lower(extra_name1) == lower(extra_name2)):

- Whatever case the packager uses in %python_extras_subpkg, it will end up in the RPM package name +suffix. For backwards compatibility, we don't change the case of the RPM package name in either way.
- The generator reads the extra name from the RPM package name +suffix, as it was put by the packager, but it must successfully find the extra even if the case differs.
- The generated provides must always provide the extras name in lowercase (regardless of the cases used in the RPM package name +suffix or the extras definition). Other replacements (e.g. of dashes, underscores, dots...) must not happen.
- All generated requires on extras must always be generated in lowercase as well to match the above. Other replacements (e.g. of dashes, underscores, dots...) must not happen.

Later:

- The %py3_dist macro must only lowercase the part in [], it should not try to canonize it as it currently does.
- We should figure out what is the correct canonical form of an extra name and make sure we obey the rules.
- We should document this in the guidelines.
Comment 4 Miro Hrončok 2021-03-09 18:01:14 UTC 
- Ensure that %pyproject_buildrequires generates BuildRequires on extras in lower case. Other replacements in extras names (e.g. of dashes, underscores, dots...) must not happen.
Comment 5 Miro Hrončok 2021-03-10 18:44:02 UTC 
In the meantime, I've started an upstream discussion about this problem: https://discuss.python.org/t/what-extras-names-are-treated-as-equal-and-why/7614
Comment 6 Tomas Orsava 2021-03-11 15:53:55 UTC 
PR: https://src.fedoraproject.org/rpms/python-rpm-generators/pull-request/35
Comment 7 Fedora Update System 2021-03-11 18:48:00 UTC 
FEDORA-2021-2712494d03 has been submitted as an update to Fedora 34. https://bodhi.fedoraproject.org/updates/FEDORA-2021-2712494d03
Comment 8 Fedora Update System 2021-03-12 18:56:08 UTC 
FEDORA-2021-2712494d03 has been pushed to the Fedora 34 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-2712494d03`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-2712494d03

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
Comment 9 Fedora Update System 2021-03-19 20:12:52 UTC 
FEDORA-2021-2712494d03 has been pushed to the Fedora 34 stable repository.
If problem still persists, please make note of it in this bug report.
Note You need to log in before you can comment on or make changes to this bug.