Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1373635 - A lot of python-bytecode-inconsistent-mtime
Summary: A lot of python-bytecode-inconsistent-mtime
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: python3
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Tomas Orsava
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-09-06 20:03 UTC by Miro Hrončok
Modified: 2018-03-01 13:04 UTC (History)
7 users (show)

Fixed In Version: python3-3.5.2-2.fc26 python3-3.5.1-15.fc24 python3-3.5.2-4.fc25
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1374667 (view as bug list)
Environment:
Last Closed: 2016-09-17 00:57:37 UTC
Type: Bug
Embargoed:

Attachments (Terms of Use)
Actual results (47.79 KB, text/plain)
2016-09-06 20:03 UTC, Miro Hrončok 		no flags Details
View All


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1550562 0 unspecified CLOSED python-bytecode-inconsistent-mtime gives bogus result with Python 3.7 bytecode 2022-05-16 11:32:56 UTC

Internal Links: 1550562

Description Miro Hrončok 2016-09-06 20:03:26 UTC 
Created attachment 1198412 [details]
Actual results

Description of problem:
When rpmlint is run on python3-libs, you get a lot of python-bytecode-inconsistent-mtime

See explanation:

$ rpmlint -I python-bytecode-inconsistent-mtime
python-bytecode-inconsistent-mtime:
The timestamp embedded in this python bytecode file isn't equal to the mtime
of the original source file, which will force the interpreter to recompile the
.py source every time, ignoring the saved bytecode.


Version-Release number of selected component (if applicable):
python3-libs-3.5.2-1.fc25

How reproducible: Easy

Steps to Reproduce:
1. rpmlint python3-libs | grep python-bytecode-inconsistent-mtime

Actual results:
Attached

Expected results:
Nothing

Additional info:
Happens since 3.5.0-1
Does not happen on 3.4
Does not happen on 2.7

With my python35 builds, it happens as well, also with python26, but not with python34, python33.
Comment 1 Miro Hrončok 2016-09-06 20:06:12 UTC 
Also happens with 3.6 from https://copr.fedorainfracloud.org/coprs/cstratak/python-3.6/
Comment 2 Charalampos Stratakis 2016-09-07 18:43:34 UTC 
It's not exclusive on python3-libs rpm. Every other subpackage that includes py and pyc files has the same warnings.

Either something is fundamentally wrong with the SPEC file (or maybe on how RPM handles some things) or it is a false positive.

Either way it needs investigation.

No idea why this happens on python26, however for Python 3.5 I guess it relates to this [PEP] somehow.

[PEP]: https://www.python.org/dev/peps/pep-0488/

So far I haven't been able to debug the exact phase of where the timestamps change, during the creation of the rpm.
Comment 3 Fedora Blocker Bugs Application 2016-09-08 08:41:02 UTC 
Proposed as a Blocker for 25-beta by Fedora user churchyard using the blocker tracking app because:

 Upgrade form F23 to F25 is not working with default Workstation install. "For each one of the release-blocking package sets, it must be possible to successfully complete a direct upgrade from fully updated installations of the last two stable Fedora releases with that package set installed. The release-blocking package sets are the minimal set, and the sets for each one of the release-blocking desktops, on all primary Architectures except i686."
Comment 4 Miro Hrončok 2016-09-08 08:47:24 UTC 
That was a mistake, I had bad bug number in clipboard, sorry about that.
Comment 5 Tomas Orsava 2016-09-08 16:43:33 UTC 
I have found out what's causing it, it's 3 different commands in the %install section that modify the files:

1. # Fix end-of-line encodings:

2. # Switch all shebangs to refer to the specific Python version.

3. # Remove shebang lines from .py files that aren't executable, and
# remove executability from .py files that don't have a shebang line:


I'm getting close to a fix.
Comment 6 Miro Hrončok 2016-09-08 17:09:24 UTC 
# Do bytecompilation with the newly installed interpreter

happens after all of those.
Comment 7 Charalampos Stratakis 2016-09-09 08:56:29 UTC 
Please report any findings here. If someone figures it out I would like the fix to be ideally pushed together with rhbz#1374357
Comment 8 Tomas Orsava 2016-09-09 10:45:37 UTC 
(In reply to Miro Hrončok from comment #6)
> # Do bytecompilation with the newly installed interpreter
> 
> happens after all of those.

Thank you, Miro, for that suggestion, it saved me a lot of time.

So, here's what was wrong: The aforementioned 3 commands in the %install section were modifying the .py files, thus the .pyc files need to be rebuilt.

Indeed the "# Do bytecompilation with the newly installed interpreter" part comes after and should rebuild all the .pyc files, however, it only rebuilds .pyc files with "optimize=0". It leaves out .pyc files with optimization level 1 and 2, which didn't used to exist, but now they do:

https://www.python.org/dev/peps/pep-0488/

I modified the script to rebuild .pyc files of all optimization levels and the problem is solved.
Comment 9 Miro Hrončok 2016-09-09 11:00:20 UTC 
In python26 the steps (fix shebangs, bytecompile) wre in bad order, trying to rebuild using the correct order.
Comment 10 Miro Hrončok 2016-09-09 11:15:14 UTC 
(In reply to Miro Hrončok from comment #9)
> In python26 the steps (fix shebangs, bytecompile) wre in bad order, trying
> to rebuild using the correct order.

Didn't help.
Comment 11 Tomas Orsava 2016-09-09 16:08:16 UTC 
Fix is verified to work and pushed to git for F24, F25 and rawhide. cstratak will make builds later after he finishes his issue.
Comment 12 Fedora Update System 2016-09-12 16:18:48 UTC 
python3-3.5.2-3.fc25 has been submitted as an update to Fedora 25. https://bodhi.fedoraproject.org/updates/FEDORA-2016-e495d8af9a
Comment 13 Fedora Update System 2016-09-12 16:20:07 UTC 
python3-3.5.1-15.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2016-80846dd659
Comment 14 Fedora Update System 2016-09-14 02:52:51 UTC 
python3-3.5.1-15.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-80846dd659
Comment 15 Fedora Update System 2016-09-14 03:25:39 UTC 
python3-3.5.2-3.fc25 has been pushed to the Fedora 25 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-e495d8af9a
Comment 16 Fedora Update System 2016-09-14 14:08:36 UTC 
python3-3.5.2-4.fc25 has been submitted as an update to Fedora 25. https://bodhi.fedoraproject.org/updates/FEDORA-2016-d49f8ec161
Comment 17 Fedora Update System 2016-09-14 19:19:01 UTC 
python3-3.5.1-15.fc24 has been pushed to the Fedora 24 stable repository. If problems still persist, please make note of it in this bug report.
Comment 18 Fedora Update System 2016-09-16 01:25:29 UTC 
python3-3.5.2-4.fc25 has been pushed to the Fedora 25 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-d49f8ec161
Comment 19 Fedora Update System 2016-09-17 00:57:30 UTC 
python3-3.5.2-4.fc25 has been pushed to the Fedora 25 stable repository. If problems still persist, please make note of it in this bug report.
Note You need to log in before you can comment on or make changes to this bug.