Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1953910 - Since rpm-4.16.90-0.git15395, python-srpm-macros in Lua are no longer found
Summary: Since rpm-4.16.90-0.git15395, python-srpm-macros in Lua are no longer found
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: rpm
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
Assignee: Packaging Maintenance Team
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: PYTHON3.10 RPM4.17
TreeView+ depends on / blocked
 
Reported: 2021-04-27 07:24 UTC by Miro Hrončok
Modified: 2021-05-05 15:15 UTC (History)
9 users (show)

Fixed In Version: rpm-4.16.90-0.git15395.3.fc35
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-04-27 15:00:57 UTC
Type: Bug
Embargoed:

Attachments (Terms of Use)
A diff between old and new `rpm --showrc` in an otherwise identical mock (20.81 KB, patch)
2021-04-27 07:35 UTC, Miro Hrončok 		no flags Details | Diff
View All


Links
System ID Private Priority Status Summary Last Updated
Github rpm-software-management rpm issues 1659 0 None open Macro file comment loading "regression" 2021-04-27 08:29:51 UTC

Description Miro Hrončok 2021-04-27 07:24:13 UTC 
Install python-srpm-macros-3.9-37.fc35 on rawhide.

The macro file is /usr/lib/rpm/macros.d/macros.python-srpm

With rpm-4.16.1.3-1.fc35.x86_64, I get:

# rpm --eval %pypi_source
https://files.pythonhosted.org/packages/source/%/%name/%name-%version.tar.gz

# rpm --eval '%py3_dist foo'
python3dist(foo) 

# rpm --eval '%py_dist_name foo_bar'
foo-bar

# rpm --eval %python_extras_subpkg
error: %python_extras_subpkg: missing option -n (name of the base package)
error: lua script failed: [string "<lua>"]:12: error expanding macro


With rpm-4.16.90-0.git15395.2.fc35.x86_64, I get:

# rpm --eval %pypi_source
%pypi_source

# rpm --eval '%py3_dist foo'
%py3_dist foo

# rpm --eval '%py_dist_name foo_bar'
%py_dist_name foo_bar

# rpm --eval %python_extras_subpkg
%python_extras_subpkg

This makes at least 1000 Fedora packages fails to build from source (most likely fail to build the SRPM) with errors like:

error: Bad source: /builddir/build/SOURCES/%{pypi_source}: No such file or directory
Comment 1 Miro Hrončok 2021-04-27 07:27:52 UTC 
Non-Lua macros from the same file work as before:

# rpm --eval %python3
/usr/bin/python3

# rpm --eval %python3_pkgversion
3


Lua macros from different files still work:

# rpm --eval '%pycached %{python3_sitelib}/foo.py'  # from /usr/lib/rpm/macros.d/macros.python3
/usr/lib/python3.9/site-packages/foo.py
/usr/lib/python3.9/site-packages/__pycache__/foo.cpython-39{,.opt-?}.pyc
Comment 2 Miro Hrončok 2021-04-27 07:32:00 UTC 
This blocks our Python 3.10 rebuild.


It can also make packages build fine, but suddenly no longer build a subpackage with https://fedoraproject.org/wiki/Changes/PythonExtras -- but only for packages that only use (conditional) %python_extras_subpkg and not %pypi_source or %py3_dist.
Comment 3 Miro Hrončok 2021-04-27 07:35:41 UTC 
Created attachment 1775859 [details]
A diff between old and new `rpm --showrc` in an otherwise identical mock

I've attached a diff between old and new `rpm --showrc` in an otherwise identical mock.
Comment 4 Panu Matilainen 2021-04-27 07:50:57 UTC 
Yup, easily reproduced. 

It's a regression introduced in https://github.com/rpm-software-management/rpm/commit/75275a87cff04da65d3557f2c40ea2b526528c4c and what it trips over is this comment line in that macro file:

>  Set to /bin/true to avoid %ifdefs and %{? in specfiles

I'll revert the patch in rawhide to let builds continue and look for the proper fix afterwards. Just a sec.
Comment 5 Miro Hrončok 2021-04-27 07:53:38 UTC 
I've just debugged that macros above `# Set to /bin/true to avoid %ifdefs and %{? in specfiles` work fine and macros below don't. You were faster


Should I escape the % in the comment? That sound s safer.
Comment 6 Miro Hrončok 2021-04-27 08:01:25 UTC 
Like this: https://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/97
Comment 7 Panu Matilainen 2021-04-27 08:06:23 UTC 
It's a good question.

In *spec* comments, macros are always expanded, but I don't recall offhand what should happen in macro file comments.

Feel free to escape though, it's probably the best immediate fix. There is a bug in rpm nevertheless (either the %{ should be ignored in a comment or there should be an error for missing }).
Comment 8 Panu Matilainen 2021-04-27 08:29:28 UTC 
Building new rpm with the problematic patch reverted now: https://koji.fedoraproject.org/koji/taskinfo?taskID=66775049
Comment 9 Miro Hrončok 2021-04-27 15:00:57 UTC 
Thanks for the revert.

I'll also land https://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/97
Comment 10 Jason Tibbitts 2021-04-27 15:55:30 UTC 
I wonder if it's better to just use %dnl "comments" in macro files instead of worrying about escaping rules.
Comment 11 Panu Matilainen 2021-04-28 05:18:37 UTC 
%dnl will *not* work in macro files because macro files are not expanded as they're read.
Comment 12 Panu Matilainen 2021-04-30 06:26:40 UTC 
So, as discussed in the upstream ticket, this wasn't so much a regression but unearthing an ancient bug - a bunch of macros in the same file were always being skipped due to the bug, but an otherwise unrelated change which permits macro body to include empty lines dragged the bug into the spotlight.

Upstream fix was to not let comments affect parsing. I've reverted the revert, and added the upstream fix instead. Was about to close this but then it already was.
Comment 13 Fedora Update System 2021-05-05 15:15:41 UTC 
FEDORA-2021-831cfbf901 has been pushed to the Fedora 35 stable repository.
If problem still persists, please make note of it in this bug report.
Note You need to log in before you can comment on or make changes to this bug.