Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1892224
Summary: | Packaging scriptlets assume global 'arg' in Lua environment | ||
---|---|---|---|
Product: | [Fedora] Fedora | Reporter: | Panu Matilainen <pmatilai> |
Component: | java-11-openjdk | Assignee: | jiri vanek <jvanek> |
Status: | CLOSED RAWHIDE | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
Severity: | urgent | Docs Contact: | |
Priority: | urgent | ||
Version: | rawhide | CC: | ahughes, akurtako, ashundi, caolanm, gwync, jerboaa, jpazdziora, jvanek, mattias.ellert, mhroncok, robert, sanjay.ankur, spotrh |
Target Milestone: | --- | Keywords: | Reopened |
Target Release: | --- | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | Doc Type: | If docs needed, set a value | |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2021-06-14 09:52:03 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Bug Depends On: | |||
Bug Blocks: | 1890881 |
Description
Panu Matilainen
2020-10-28 09:02:53 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 34 development cycle. Changing version to 34. Rpm 4.17 is about to land in rawhide, at which point this issue will activate. You'll need to pass the 'arg' variable around now, or conditionalize access to it. Oh, it's not just updates, installs are also affected (just hadn't seen it myself) and thus all builds requiring openjdk now fail :-/ Eg https://kojipkgs.fedoraproject.org//work/tasks/4071/66714071/root.log Is this issue related copy-jdk-configs or the snippet in jdk itself? https://src.fedoraproject.org/rpms/java-11-openjdk/blob/rawhide/f/java-11-openjdk.spec#_2063 x https://pagure.io/copy_jdk_configs/blob/master/f/copy_jdk_configs.lua (In reply to Panu Matilainen from comment #2) > Rpm 4.17 is about to land in rawhide, at which point this issue will > activate. > > You'll need to pass the 'arg' variable around now, or conditionalize access > to it. I see We set it correctly: https://src.fedoraproject.org/rpms/java-11-openjdk/blob/rawhide/f/java-11-openjdk.spec#_2101 So the usage via reqires in https://pagure.io/copy_jdk_configs/blob/master/f/copy_jdk_configs.lua shoudl be ok.. well is not... Any idea how to not update all spec files of all jdks welcomed... The https://rpm.org/user_doc/lua.html is still wrong: criptlet arguments are accessible from a global ‘arg’ table. Even if "Remove global arg table from the environment" is honored, I had devlared args on my own. Still it is dropped. Hi Panu, I need a bit more info to find some woarkaround. I had redeclared the arg variable, still the rpm had dropped it. Why so? I don't recall the details of Lua module environments off-hand, but if you think about it, it makes perfect sense that loaded modules do not see variables declared by the executing program. Please change the copying to be a function which takes arguments, Lua modules are not really meant to be used like this in the first place. I get that it's pain to change all the jdk specs to this but it's a one-time change and the same construct will work in all rpm versions. https://pagure.io/copy_jdk_configs/pull-request/1 Should be the fix. It should also remain backward compatible with f34 and older fedoras, so no immediate backporting should be needed. More eyes on it welcomed. hello! I'm failing to verify my fix. If I run rawhide chroot on most updated f33 box, There is still rpm 4.16. Even if I install manually latest koji build of rpm - https://koji.fedoraproject.org/koji/buildinfo?buildID=1740980 the bug do not pop up. (In reply to jiri vanek from comment #11) > https://pagure.io/copy_jdk_configs/pull-request/1 got updated a bit + fixed rpms: https://src.fedoraproject.org/rpms/java-11-openjdk/pull-request/115 https://src.fedoraproject.org/rpms/copy-jdk-configs/pull-request/2 I had built https://koji.fedoraproject.org/koji/buildinfo?buildID=1741785 It is already in buildroot, and it had fixed the issue. As copy-jdk-configs are used for updating the openjdk's, fact that java-*-openjdk is not yet updated is irrelevant for buildroot. The pathes to openjdk will follow. (In reply to jiri vanek from comment #13) > (In reply to jiri vanek from comment #11) > > https://pagure.io/copy_jdk_configs/pull-request/1 > got updated a bit > > + fixed rpms: https://src.fedoraproject.org/rpms/java-latest-openjdk/pull-request/49 https://src.fedoraproject.org/rpms/java-1.8.0-openjdk/pull-request/134 > https://src.fedoraproject.org/rpms/java-11-openjdk/pull-request/115 > https://src.fedoraproject.org/rpms/copy-jdk-configs/pull-request/2 testing done old java + old cjc ok f34/f35 old java + new cjc ok f34/f35 new java + new cjc ok f34/f35 old java + old cjc -> new cjc -> new java ok f34/f35 old java + old cjc -> new java -> new cjc fail f34/f35 old java + old cjc -> new cjc + new java ok f34/f35 All should benow pushed ant built *** Bug 1954998 has been marked as a duplicate of this bug. *** We see errors when installing java-11-openjdk-headless into a rawhide mock chroot using rpm+dnf from Fedora 33/34: DEBUG util.py:446: Running scriptlet: java-11-openjdk-headless-1:11.0.11.0.9-2.fc35.x86_64 1/1 DEBUG util.py:446: error: lua script failed: /var/lib/rpm-state//copy_jdk_configs.lua:215: bad argument #1 to 'gsub' (string expected, got nil) DEBUG util.py:444: Error in PRETRANS scriptlet in rpm package java-11-openjdk-headless Reproducer: $ mock -r fedora-rawhide-x86_64 --no-bootstrap-chroot --init $ mock -r fedora-rawhide-x86_64 --no-bootstrap-chroot --install java-11-openjdk-headless When using the bootstrap chroot (and updating RPM in it to 4.17), the package installs just fine. When we update Python from 3.9 to 3.10, we cannot use the bootstrap chroot option, because we will not have a working dnf for a brief moment. I suppose we can: 1) disable bootstrap chroot 2) build anything up until dnf, but no java packages 3) enable bootstrap chroot 4) build the rest However, if there is some java pulled in before we can get to dnf (not sure, hopefully not), this won't work. Could you please fix the package in a way that is compatible with both RPM 4.17+ and RPM 4.16? This needs to be resolved quickly, withing 2 weeks. Another reproducible method: 1. docker run -ti fedora:rawhide /bin/bash 2. dnf install java-11-openjdk-headless 3. Failed: java-11-openjdk-headless-1:11.0.11.0.9-2.fc35.x86_64 4. dnf history info 2 Scriptlet output: 1 error: lua script failed: /var/lib/rpm-state//copy_jdk_configs.lua:215: bad argument #1 to 'gsub' (string expected, got nil) Hope that helps, thanks all! fedora-review is hitting this when it runs tests rawhide builds in mock, so this could be affecting a number of review tickets. Running this should unblock you: $ mock -r fedora-rawhide-x86_64 --scrub=bootstrap I've debugged the cause of this. This is the scriptlet: ... cjc = require "copy_jdk_configs.lua" arg = {"--currentjvm", "%{uniquesuffix %{nil}}", "--jvmdir", "%{_jvmdir %{nil}}", "--origname", "%{name}", "--origjavaver", "%{javaver}", "--arch", "%{_arch}", "--temp", "%{rpm_state_dir}/%{name}.%{_arch}"} cjc.mainProgram(arg) And this is at the end of the /usr/libexec/copy_jdk_configs.lua file: if (arg == nil) then debugOneLinePrint("arg variable is nil, you have to call mainProgram manually") -- this can actually not be invoked, as the debug is set via argz else M.mainProgram(arg) end On RPM 4.17, arg is not implicitly set and hence the require line only goes to the debugOneLinePrint() line. Later, the function is acalled directly from the scriptlet with arguments provided. On RPM 4.16, arg is (implicitly?) set to an empty table. Hence the "require" will actually run M.mainProgram({}) and that function is not handling missing arguments very well. To support both RPM 4.16 and 4.17, I suggest not to run any code when the module is required. If that is not possible due to old packages out there expecting the current weird behavior, the scriptlet can be fixed by adding `arg = nil` before the require: ... arg = nil cjc = require "copy_jdk_configs.lua" arg = {"--currentjvm", "%{uniquesuffix %{nil}}", "--jvmdir", "%{_jvmdir %{nil}}", "--origname", "%{name}", "--origjavaver", "%{javaver}", "--arch", "%{_arch}", "--temp", "%{rpm_state_dir}/%{name}.%{_arch}"} cjc.mainProgram(arg) I have tested that with bootstrap mode and without it and it installs successfully in both cases. Thanx a lot for analyse. From what I had read, the "new" error is appearing only when rpm 4.16->4.17 is in transaction. So during f34->f35. correct? All other issues (like broken buildroot or so), are fixed as I expect. Will add the `arg = nil` to the JDK rpms. The problem I've reported is not primarily about updates to Fedora 35.
It happens when you use the old RPM to install the new JDK. Which can happen in Koji, mock or copr (when certain config is set).
As reported by Robert, it also happens to users, when they have a stlightly outdated rawhide installation.
And, as you've somehow identified as well, it most likely also blocks upgrades from Fedora 33 or 34 to Fedora 35 (but I have not tried to reproduce that).
> Will add the `arg = nil` to the JDK rpms.
Thanks. That should unblock us. java-11-openjdk is the main priority for us.
https://pagure.io/copy_jdk_configs/pull-request/3#request_diff Will now distribute it over JDKS, Starting with f34 and f35 jdk11. Eyball welcomed. (In reply to jiri vanek from comment #28) > https://pagure.io/copy_jdk_configs/pull-request/3#request_diff > > Will now distribute it over JDKS, Starting with f34 and f35 jdk11. > > Eyball welcomed. With this change, the Requires: copy-jdk-configs >= 4.0 is of course necessary (In reply to jiri vanek from comment #29) > (In reply to jiri vanek from comment #28) > > https://pagure.io/copy_jdk_configs/pull-request/3#request_diff > > > > Will now distribute it over JDKS, Starting with f34 and f35 jdk11. > > > > Eyball welcomed. > > With this change, the > Requires: copy-jdk-configs >= 4.0 > is of course necessary jdk11 f34: https://src.fedoraproject.org/rpms/java-11-openjdk/pull-request/118#request_diff jdk11 f35: https://src.fedoraproject.org/rpms/java-11-openjdk/pull-request/117#request_diff The relevant bits ar enow built in koji. Updates posted |