Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1719647
Summary: | rdoc gzipped javascript pages are not the same across multilib | ||
---|---|---|---|
Product: | Red Hat Enterprise Linux 8 | Reporter: | Richard W.M. Jones <rjones> |
Component: | ruby | Assignee: | Jun Aruga <jaruga> |
Status: | CLOSED ERRATA | QA Contact: | David Jež <djez> |
Severity: | unspecified | Docs Contact: | |
Priority: | unspecified | ||
Version: | 8.2 | CC: | djez, jaruga, ptoscano, vondruch |
Target Milestone: | rc | ||
Target Release: | 8.0 | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | ruby-2.5.5-105.module+el8.1.0+3656+f80bfa1d | Doc Type: | No Doc Update |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2019-11-05 20:57:22 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Bug Depends On: | |||
Bug Blocks: | 910269 |
Description
Richard W.M. Jones
2019-06-12 09:14:42 UTC
(In reply to Richard W.M. Jones from comment #0) > /usr/share/doc/ruby-hivex/api/created.rid Note this is not a gzip'ed js file, although it changes across architectures. (In reply to Pino Toscano from comment #1) > (In reply to Richard W.M. Jones from comment #0) > > /usr/share/doc/ruby-hivex/api/created.rid > > Note this is not a gzip'ed js file, although it changes across architectures. Yes this one is likely a separate bug. The file contains timestamps which are all the same for source files, but then there's another timestamp which is different although it's not clear what it refers to: $ cat ./i686/usr/share/doc/ruby-hivex/api/created.rid Sat, 25 May 2019 04:07:14 +0000 ./README.rdoc Mon, 03 Dec 2012 16:05:45 +0000 ./lib/hivex.rb Mon, 03 Dec 2012 16:05:45 +0000 ./ext/hivex/_hivex.c Mon, 26 Feb 2018 10:50:23 +0000 ./ext/hivex/extconf.h Sat, 25 May 2019 04:07:14 +0000 $ cat x86_64//usr/share/doc/ruby-hivex/api/created.rid Sat, 25 May 2019 04:07:09 +0000 ./README.rdoc Mon, 03 Dec 2012 16:05:45 +0000 ./lib/hivex.rb Mon, 03 Dec 2012 16:05:45 +0000 ./ext/hivex/_hivex.c Mon, 26 Feb 2018 10:50:23 +0000 ./ext/hivex/extconf.h Sat, 25 May 2019 04:07:09 +0000 See also: https://bugs.ruby-lang.org/issues/13627 We could probably just rm this file. And this is probably workaround for the .gz: https://github.com/ruby/rdoc/pull/569 Hm, so we have RDoc 6.0.1 while both these issues were resolved/workarounded in 6.0.2, but you still would need to use the ENV variable. Not sure what we can do about this issue in RHEL. But the env variable usage feels as a material for Fedora ... (In reply to Vít Ondruch from comment #4) > Hm, so we have RDoc 6.0.1 while both these issues were resolved/workarounded > in 6.0.2, but you still would need to use the ENV variable. Not sure what we > can do about this issue in RHEL. It looks like both the patches use the SOURCE_DATE_EPOCH approach of reproducible builds. It is easy enough to use it in a spec file -- for example Jirka Denemar and me came out with export SOURCE_DATE_EPOCH=$(stat --printf='%Y' %{_specdir}/%{name}.spec) for libvirt. > But the env variable usage feels as a material for Fedora ... Not sure, what do you mean with that? Shall we assign this to hivex to add that line as suggested by Pino, and duplicate it for several products: Fedora / hivex Fedora / libguestfs RHEL 8 / libguestfs ? Hit "Save" too early... Vít: if you backport the two patches (or rebase rdoc), and reassign this bug to rdoc, I can clone this for hivex in RHEL & RHEL AV, and fix it once the fixed rdoc is available. What do you think? (In reply to Pino Toscano from comment #5) > (In reply to Vít Ondruch from comment #4) > > Hm, so we have RDoc 6.0.1 while both these issues were resolved/workarounded > > in 6.0.2, but you still would need to use the ENV variable. Not sure what we > > can do about this issue in RHEL. > > It looks like both the patches use the SOURCE_DATE_EPOCH approach of > reproducible builds. > It is easy enough to use it in a spec file -- for example Jirka Denemar and > me came out with > export SOURCE_DATE_EPOCH=$(stat --printf='%Y' %{_specdir}/%{name}.spec) > for libvirt. But this is not supported by: ~~~ Version-Release number of selected component (if applicable): rubygem-rdoc-6.0.1-103.module+el8+2671+ebcc7ee0.noarch ~~~ > > But the env variable usage feels as a material for Fedora ... > > Not sure, what do you mean with that? Not sure how the noarch packages are processed on Fedora, but I assume that without the env variable, we have to face such issues in Fedora at some circumstances. And it would be probably useful to set the env varaible by default. But this should be developed and tested in Fedora first. (In reply to Vít Ondruch from comment #8) > (In reply to Pino Toscano from comment #5) > > (In reply to Vít Ondruch from comment #4) > > > Hm, so we have RDoc 6.0.1 while both these issues were resolved/workarounded > > > in 6.0.2, but you still would need to use the ENV variable. Not sure what we > > > can do about this issue in RHEL. > > > > It looks like both the patches use the SOURCE_DATE_EPOCH approach of > > reproducible builds. > > It is easy enough to use it in a spec file -- for example Jirka Denemar and > > me came out with > > export SOURCE_DATE_EPOCH=$(stat --printf='%Y' %{_specdir}/%{name}.spec) > > for libvirt. > > But this is not supported by: > > ~~~ > Version-Release number of selected component (if applicable): > > rubygem-rdoc-6.0.1-103.module+el8+2671+ebcc7ee0.noarch > ~~~ Yes, I know, I meant that it will be easy to use SOURCE_DATE_EPOCH once the patches/fixes are available in rdoc. > > > But the env variable usage feels as a material for Fedora ... > > > > Not sure, what do you mean with that? > > Not sure how the noarch packages are processed on Fedora, but I assume that > without the env variable, we have to face such issues in Fedora at some > circumstances. And it would be probably useful to set the env varaible by > default. But this should be developed and tested in Fedora first. I don't see these problems in the Fedora packaging, so I assume it is not needed there. When the patches are applied, the reproducer is as easy as: ~~~ $ cd into/ruby/spec/directory $ rdoc test_systemtap.rb # or any other .rb file ... snip ... $ md5sum doc/created.rid doc/js/*.js.gz 5cc8afe1bdfeca5abd9c53c7be0aff02 doc/created.rid 6675f6a42a0685d5feb02c3802c4311e doc/js/navigation.js.gz cb45ba52509d3c1486f93594c3577481 doc/js/searcher.js.gz e12315987bf7a8525658cac12b41165c doc/js/search_index.js.gz $ rdoc test_systemtap.rb # or any other .rb file ... snip ... $ md5sum doc/created.rid doc/js/*.js.gz 86f57716613a1a7bb519692c672435b2 doc/created.rid 6675f6a42a0685d5feb02c3802c4311e doc/js/navigation.js.gz cb45ba52509d3c1486f93594c3577481 doc/js/searcher.js.gz 6830b12fa4a33b34d0954243eff4b4db doc/js/search_index.js.gz $ # Checksums are modified due to timestamps $ SOURCE_DATE_EPOCH=$(stat --printf='%Y' ruby.spec) rdoc test_systemtap.rb ... snip ... $ md5sum doc/created.rid doc/js/*.js.gz b64541e4046243f4272aee9749ac4fc4 doc/created.rid 6675f6a42a0685d5feb02c3802c4311e doc/js/navigation.js.gz cb45ba52509d3c1486f93594c3577481 doc/js/searcher.js.gz 03852b912f354db9e92d4bc79479558e doc/js/search_index.js.gz $ SOURCE_DATE_EPOCH=$(stat --printf='%Y' ruby.spec) rdoc test_systemtap.rb ... snip ... $ md5sum doc/created.rid doc/js/*.js.gz b64541e4046243f4272aee9749ac4fc4 doc/created.rid 6675f6a42a0685d5feb02c3802c4311e doc/js/navigation.js.gz cb45ba52509d3c1486f93594c3577481 doc/js/searcher.js.gz 03852b912f354db9e92d4bc79479558e doc/js/search_index.js.gz $ # Checksums are stable, because the predefined timestamp is used. ~~~ I thinking to include this PR along the upstream patches: https://src.fedoraproject.org/rpms/ruby/pull-request/47 This sets the SOURCE_DATE_EPOCH for every rubygem- package. It won't help this particular case, but others might hopefully benefit. Note: this will be fixed new version redhat-rpm-config. We do not need to change ruby. https://src.fedoraproject.org/rpms/ruby/pull-request/47#comment-26983 https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/57 (In reply to Jun Aruga from comment #12) > Note: this will be fixed new version redhat-rpm-config. > We do not need to change ruby. This is RHEL ticket not Fedora * So unless somebody is going to request fix for redhat-rpm-config in RHEL, the SOURCE_DATE_EPOCH is not going to be set by rpmbuild * The Ruby 2.5 does not have the rubygem-rdoc up2date, so the patches needs to be applied. Just summarizing this issue. If rubygem-rdoc < v6.0.2, need below patch. https://github.com/ruby/rdoc/pull/569 rubygem-rdoc versions in rpms/ruby * stream-ruby-2.6-rhel-8.1.0 branch: 6.1.0 * stream-ruby-2.5-rhel-8.1.0 branch: 6.0.1 (In reply to Jun Aruga from comment #14) > If rubygem-rdoc < v6.0.2, need below patch. > https://github.com/ruby/rdoc/pull/569 This ^^ is incorrect. Let me fix this for you: If rubygem-rdoc < v6.0.2, need below patches. https://github.com/ruby/rdoc/pull/570 https://github.com/ruby/rdoc/pull/569 I'm a bit cnfused about where we are with this bug. We can certainly patch hivex to fix this (just for hivex of course); if so pleae reassign the bug to hivex. But it looks as if to fix this "properly" we'll need to patch redhat-rpm-config. Contrary to claims here, it's perfectly possible for us to do this, we will need to file a bug against redhat-rpm-config. I assume (correctly?) that if we fix redhat-rpm-config then no change will be needed to hivex. > if we fix redhat-rpm-config then no change will be needed to hivex. I think it depends what you want to fix. Right now the logic to create rdoc document in hivex is in the process of "make". https://src.osci.redhat.com/rpms/hivex/blob/stream-rhel-rhel-8.1.0/f/hivex.spec#_191 ``` + make V=1 INSTALLDIRS=vendor -j4 ... rake rdoc Parsing sources... 25% [ 1/ 4] README.rdoc 50% [ 2/ 4] ext/hivex/_hivex.c 75% [ 3/ 4] ext/hivex/extconf.h 100% [ 4/ 4] lib/hivex.rb BUILDSTDERR: Generating Darkfish format into /builddir/build/BUILD/hivex-1.3.15/ruby/doc/site/api... Files: 4 Classes: 2 (2 undocumented) Modules: 1 (1 undocumented) Constants: 0 (0 undocumented) Attributes: 0 (0 undocumented) Methods: 30 (0 undocumented) Total: 33 (3 undocumented) 90.91% documented Elapsed: 0.2s ``` Below Rakefile.in is used for the "rake rdoc". https://github.com/libguestfs/hivex/blob/v1.3.15/ruby/Rakefile.in#L79-L83 ``` RDOC_FILES = FileList[ "@srcdir@/README.rdoc", "@srcdir@/lib/**/*.rb", "@srcdir@/ext/**/*.[ch]" ] ``` We tried to set a value of SOURCE_DATE_EPOCH in %gem_install macro in rpms/ruby ruby.spec that is used to install rubygem-* RPM package. But in case of ruby-hivex, it is not used. I suppose that is the reason SOURCE_DATE_EPOCH is not set for hte ruby-hivex's rdoc document gz file's meta data (?). If we can fix redhat-rpm-config like Fedora rawhide to set SOURCE_DATE_EPOCH value, you do not have to add below kind of hivex.spec before "make". Otherwise you need to add the logic. ``` export SOURCE_DATE_EPOCH=$(stat --printf='%Y' %{SOURCE0}) ``` Let me debug a little bit more. I want to see what happens when set "export SOURCE_DATE_EPOCH" in hivex.spec. (In reply to Richard W.M. Jones from comment #21) > But it looks as if to fix this "properly" we'll need to patch redhat-rpm-config. > Contrary to claims here, it's perfectly possible for us to do this, we will need to > file a bug against redhat-rpm-config. That change is not without issues, since there is still ongoing discussion in the Fedora ticket: https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/57 > I assume (correctly?) that if we fix > redhat-rpm-config then no change will be needed to hivex. Right. > ``` > export SOURCE_DATE_EPOCH=$(stat --printf='%Y' %{SOURCE0}) > ``` > > Let me debug a little bit more. I want to see what happens when set "export SOURCE_DATE_EPOCH" in hivex.spec. * searcher.js.gz : OK as we know. * search_index.js.gz : OK as SOURCE_DATE_EPOCH in hivex.spec is set correctly. * navigation.js.gz : NOT OK, the reason is ruby copies the file navigation.js in the template directory to installed directory. But in the ruby.spec, we are patching navigation.js file with system date. That is a bug. * created.rid : NOT OK. Because 1st line of the file is updated as a value of SOURCE_DATE_EPOCH. But it seems that actually extconf.h is generated with system date in the make process of hivex. That causes the difference between each files. ``` $ diff ./new_hivex_set_source_date/result.x86_64/usr/share/doc/ruby-hivex/api/created.rid ./new_hivex_set_source_date/result.i686/usr/share/doc/ruby-hivex/api/created.rid 5c5 < ./ext/hivex/extconf.h Wed, 10 Jul 2019 15:11:30 +0200 --- > ./ext/hivex/extconf.h Wed, 10 Jul 2019 15:18:14 +0200 $ cat ./new_hivex_set_source_date/result.x86_64/usr/share/doc/ruby-hivex/api/created.rid Fri, 04 May 2018 21:38:37 -0000 <= This line is ok. ./README.rdoc Mon, 03 Dec 2012 17:05:45 +0100 ./lib/hivex.rb Mon, 03 Dec 2012 17:05:45 +0100 ./ext/hivex/_hivex.c Mon, 26 Feb 2018 11:50:23 +0100 ./ext/hivex/extconf.h Wed, 10 Jul 2019 15:11:30 +0200 ``` For created.rid, you can patch like this. ``` $ git diff diff --git a/ruby/Rakefile.in b/ruby/Rakefile.in index cd99507..83125bb 100644 --- a/ruby/Rakefile.in +++ b/ruby/Rakefile.in @@ -76,11 +76,14 @@ Rake::TestTask.new(:test) do |t| end task :test => :build +CREATED_FILES = [ + "@srcdir@/ext/hivex/extconf.h" +] RDOC_FILES = FileList[ "@srcdir@/README.rdoc", "@srcdir@/lib/**/*.rb", "@srcdir@/ext/**/*.[ch]" -] +] - CREATED_FILES Rake::RDocTask.new do |rd| rd.main = "@srcdir@/README.rdoc" ``` > * navigation.js.gz : NOT OK, the reason is ruby copies the file navigation.js in the template directory to installed directory. But in the ruby.spec, we are patching navigation.js file with system date. That is a bug.
I fixed navigation.js.gz in rpms/ruby.
```
$ find . -name "navigation.js.gz" | xargs md5sum
8e4ae8f305b2a1cd808057bc1db70253 ./result.x86_64/usr/share/doc/ruby-hivex/api/js/navigation.js.gz
8e4ae8f305b2a1cd808057bc1db70253 ./result.i686/usr/share/doc/ruby-hivex/api/js/navigation.js.gz
```
So, summarizing, Steps hivex can do on is below.
1. Wait rpms/ruby 2.5 new release applying patches.
2. hivex can set export SOURCE_DATE_EPOCH in hivex.spec or wait redhat-rpm-config fixing it.
3. hivex can apply above patch for created.rid to fix the issue, or waive it as false positive.
> 1. Wait rpms/ruby 2.5 new release applying patches.
It was fixed at ruby-2.5.5-105.module+el8.1.0+3656+f80bfa1d in modules/ruby 2.5
QA team for rpms/ruby, I show you how to test rpms/ruby for this ticket. This ticket's issue is below files in ruby-hivex binary RPM are different between x86_64 and i686 RPM file. > https://bugzilla.redhat.com/show_bug.cgi?id=1719647#c0 > https://rpmdiff.engineering.redhat.com/run/407212/18/ /usr/share/doc/ruby-hivex/api/js/searcher.js.gz /usr/share/doc/ruby-hivex/api/js/search_index.js.gz /usr/share/doc/ruby-hivex/api/js/navigation.js.gz /usr/share/doc/ruby-hivex/api/created.rid searcher.js.gz, navigation.js.gz, search_index.js.gz and created.rid are created at the build time of ruby-hivex. The differences are searcher.js.gz and navigation.js.gz are created as below steps. 1. Copy ./usr/share/gems/gems/rdoc-6.0.1/lib/rdoc/generator/template/json_index/js/searcher.js in rubygem-rdoc-*.noarch RPM to /usr/share/doc/ruby-hivex/api/js/searcher.js 2. searcher.js.gz is created from /usr/share/doc/ruby-hivex/api/js/searcher.js using the /usr/share/doc/ruby-hivex/api/js/searcher.js 's modified date as a searcher.js.gz file's metadata. So, when /usr/share/doc/ruby-hivex/api/js/searcher.js's modified date is not fixed value, the created gz file's content is different. This ticket's patch is to make below copy preserving the file's modified date like "cp -p". As a result, the created gz file has fixed content between x86_64 and i686 (architectures). > 1. Copy ./usr/share/gems/gems/rdoc-6.0.1/lib/rdoc/generator/template/json_index/js/searcher.js in rubygem-rdoc-*.noarch RPM to /usr/share/doc/ruby-hivex/api/js/searcher.js navigation.js.gz is same as well as searcher.js.gz. I also preserved the modified date for ./usr/share/gems/gems/rdoc-6.0.1/lib/rdoc/generator/template/json_index/js/navigation.js . Last time navigation.js was updated as system date at the build time of rpms/ruby. But this modification was not necessary. I made mistake. Because rubygem-rdoc RPM is noarch package. The common rubygem-rdoc is used by each arch's ruby-hivex. It does not matter that a file in rubygem-rdoc RPM is updated as system date or not. Sorry for that. I tested this ticket with wrong way. Though it's not harmful. search_index.js.gz are created as below steps. 1. ruby-hivex's search_index.js is newly created at build time of ruby-hivex. 2. The search_index.js's modified date is updated as a value of a environment variable SOURCE_DATE_EPOCH if SOURCE_DATE_EPOCH exists. 3. search_index.js.gz is created from the search_index.js using the search_index.js's modified date as the gz file's metadata. created.rid is created at build time of ruby-hivex. 1st line (date string) in created.rid is updated as string of SOURCE_DATE_EPOCH if SOURCE_DATE_EPOCH exists. Otherwise it is updated as system date. So, QA team. There are 2 tests to verify this ticket for rpms/ruby. 1. Check the result of build.log. If below tests are passed, it's okay. http://download.eng.bos.redhat.com/brewroot/vol/rhel-8/packages/ruby/2.5.5/105.module+el8.1.0+3656+f80bfa1d/data/logs/x86_64/build.log > [11694/17422] TestRDocGeneratorJsonIndex#test_generate = 0.00 s > [11695/17422] TestRDocGeneratorJsonIndex#test_generate_search_index_with_reproducible_builds = 0.00 s 2. Check rubygem-rdoc RPM. Check below files are not updated as system date (July 12th 2019), that is okay, but preserved as the original file's modified date. https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=929456 $ curl -O http://download.eng.bos.redhat.com/brewroot/vol/rhel-8/packages/ruby/2.5.5/105.module+el8.1.0+3656+f80bfa1d/noarch/rubygem-rdoc-6.0.1-105.module+el8.1.0+3656+f80bfa1d.noarch.rpm Extract the RPM file. ``` $ find . -name searcher.js | xargs ls -l -rw-r--r-- 1 jaruga jaruga 6633 Sep 5 2016 ./usr/share/gems/gems/rdoc-6.0.1/lib/rdoc/generator/template/json_index/js/searcher.js $ find . -name navigation.js | xargs ls -l -rw-r--r-- 1 jaruga jaruga 3652 Nov 27 2012 ./usr/share/gems/gems/rdoc-6.0.1/lib/rdoc/generator/template/json_index/js/navigation.js ``` For rpms/ruby-hivex, I have already wrote how to modify. https://bugzilla.redhat.com/show_bug.cgi?id=1719647#c26 > So, summarizing, steps hivex can do on is below. > > 1. Wait rpms/ruby 2.5 new release applying patches. > 2. hivex can set export SOURCE_DATE_EPOCH in hivex.spec or wait redhat-rpm-config fixing it. > 3. hivex can apply above patch for created.rid to fix the issue, or waive it as false positive. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:3384 |