Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1374667 - A lot of python-bytecode-inconsistent-mtime
Summary: A lot of python-bytecode-inconsistent-mtime
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Software Collections
Classification: Red Hat
Component: python
Version: rh-python35
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: beta
: 2.3
Assignee: Tomas Orsava
QA Contact: Branislav Náter
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-09-09 11:20 UTC by Tomas Orsava
Modified: 2016-11-15 10:00 UTC (History)
10 users (show)

Fixed In Version: rh-python35-python-3.5.1-11.el7 rh-python35-python-3.5.1-11.el6
Doc Type: No Doc Update
Doc Text:
undefined
Clone Of: 1373635
Environment:
Last Closed: 2016-11-15 10:00:24 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2016:2720 0 normal SHIPPED_LIVE rh-python35 bug fix and enhancement update 2016-11-15 14:28:12 UTC

Description Tomas Orsava 2016-09-09 11:20:40 UTC 
+++ This bug was initially created as a clone of Bug #1373635 +++

Description of problem:
When rpmlint is run on python3-libs, you get a lot of python-bytecode-inconsistent-mtime

See explanation:

$ rpmlint -I python-bytecode-inconsistent-mtime
python-bytecode-inconsistent-mtime:
The timestamp embedded in this python bytecode file isn't equal to the mtime
of the original source file, which will force the interpreter to recompile the
.py source every time, ignoring the saved bytecode.


Version-Release number of selected component (if applicable):
python3-libs-3.5.2-1.fc25

How reproducible: Easy

Steps to Reproduce:
1. rpmlint python3-libs | grep python-bytecode-inconsistent-mtime

Actual results:
Attached

Expected results:
Nothing

Additional info:
Happens since 3.5.0-1
Does not happen on 3.4
Does not happen on 2.7

With my python35 builds, it happens as well, also with python26, but not with python34, python33.

--- Additional comment from Miro Hrončok on 2016-09-06 22:06:12 CEST ---

Also happens with 3.6 from https://copr.fedorainfracloud.org/coprs/cstratak/python-3.6/

--- Additional comment from Charalampos Stratakis on 2016-09-07 20:43:34 CEST ---

It's not exclusive on python3-libs rpm. Every other subpackage that includes py and pyc files has the same warnings.

Either something is fundamentally wrong with the SPEC file (or maybe on how RPM handles some things) or it is a false positive.

Either way it needs investigation.

No idea why this happens on python26, however for Python 3.5 I guess it relates to this [PEP] somehow.

[PEP]: https://www.python.org/dev/peps/pep-0488/

So far I haven't been able to debug the exact phase of where the timestamps change, during the creation of the rpm.

--- Additional comment from Fedora Blocker Bugs Application on 2016-09-08 10:41:02 CEST ---

Proposed as a Blocker for 25-beta by Fedora user churchyard using the blocker tracking app because:

 Upgrade form F23 to F25 is not working with default Workstation install. "For each one of the release-blocking package sets, it must be possible to successfully complete a direct upgrade from fully updated installations of the last two stable Fedora releases with that package set installed. The release-blocking package sets are the minimal set, and the sets for each one of the release-blocking desktops, on all primary Architectures except i686."

--- Additional comment from Miro Hrončok on 2016-09-08 10:47:24 CEST ---

That was a mistake, I had bad bug number in clipboard, sorry about that.

--- Additional comment from Tomas Orsava on 2016-09-08 18:43:33 CEST ---

I have found out what's causing it, it's 3 different commands in the %install section that modify the files:

1. # Fix end-of-line encodings:

2. # Switch all shebangs to refer to the specific Python version.

3. # Remove shebang lines from .py files that aren't executable, and
# remove executability from .py files that don't have a shebang line:


I'm getting close to a fix.

--- Additional comment from Miro Hrončok on 2016-09-08 19:09:24 CEST ---

# Do bytecompilation with the newly installed interpreter

happens after all of those.

--- Additional comment from Charalampos Stratakis on 2016-09-09 10:56:29 CEST ---

Please report any findings here. If someone figures it out I would like the fix to be ideally pushed together with rhbz#1374357

--- Additional comment from Tomas Orsava on 2016-09-09 12:45:37 CEST ---

(In reply to Miro Hrončok from comment #6)
> # Do bytecompilation with the newly installed interpreter
> 
> happens after all of those.

Thank you, Miro, for that suggestion, it saved me a lot of time.

So, here's what was wrong: The aforementioned 3 commands in the %install section were modifying the .py files, thus the .pyc files need to be rebuilt.

Indeed the "# Do bytecompilation with the newly installed interpreter" part comes after and should rebuild all the .pyc files, however, it only rebuilds .pyc files with "optimize=0". It leaves out .pyc files with optimization level 1 and 2, which didn't used to exist, but now they do:

https://www.python.org/dev/peps/pep-0488/

I modified the script to rebuild .pyc files of all optimization levels and the problem is solved.

--- Additional comment from Miro Hrončok on 2016-09-09 13:00:20 CEST ---

In python26 the steps (fix shebangs, bytecompile) wre in bad order, trying to rebuild using the correct order.

--- Additional comment from Miro Hrončok on 2016-09-09 13:15:14 CEST ---

(In reply to Miro Hrončok from comment #9)
> In python26 the steps (fix shebangs, bytecompile) wre in bad order, trying
> to rebuild using the correct order.

Didn't help.
Comment 2 Tomas Orsava 2016-09-09 12:32:22 UTC 
This bug was erroneously filed for the metapackage component, correcting my mistake.
Comment 6 Branislav Náter 2016-10-26 13:58:34 UTC 
rpmlint version used: rpmlint-1.9-3.fc25

Reproduced on rhscl-2.2 packages:
# for file in *.rpm; do echo $file; rpmlint $file | grep python-bytecode-inconsistent-mtime; done
rh-python35-python-devel-3.5.1-9.el7.x86_64.rpm
rh-python35-python-tkinter-3.5.1-9.el7.x86_64.rpm
rh-python35-python-tkinter.x86_64: E: python-bytecode-inconsistent-mtime /opt/rh/rh-python35/root/usr/lib64/python3.5/tkinter/__pycache__/commondialog.cpython-35.opt-2.pyc 2015-12-06T20:39:09 /opt/rh/rh-python35/root/usr/lib64/python3.5/tkinter/commondialog.py 2016-08-06T06:53:24
<snip>
rh-python35-python-3.5.1-9.el7.x86_64.rpm
rh-python35-python-docutils-0.12-1.el7.noarch.rpm
rh-python35-python-tools-3.5.1-9.el7.x86_64.rpm
rh-python35-python-debug-3.5.1-9.el7.x86_64.rpm
rh-python35-python-debug.x86_64: E: python-bytecode-inconsistent-mtime /opt/rh/rh-python35/root/usr/lib64/python3.5/config-3.5dm/__pycache__/python-config.cpython-35.opt-2.pyc 2016-08-06T06:51:13 /opt/rh/rh-python35/root/usr/lib64/python3.5/config-3.5dm/python-config.py 2016-08-06T06:53:23
rh-python35-python-libs-3.5.1-9.el7.x86_64.rpm
rh-python35-python-libs.x86_64: E: python-bytecode-inconsistent-mtime /opt/rh/rh-python35/root/usr/lib64/python3.5/__pycache__/pickle.cpython-35.opt-2.pyc 2015-12-06T20:39:08 /opt/rh/rh-python35/root/usr/lib64/python3.5/pickle.py 2016-08-06T06:53:20
<snip>
rh-python35-python-virtualenv-13.1.2-1.el7.noarch.rpm
rh-python35-python-debuginfo-3.5.1-9.el7.x86_64.rpm
rh-python35-python-test-3.5.1-9.el7.x86_64.rpm
rh-python35-python-test.x86_64: E: python-bytecode-inconsistent-mtime /opt/rh/rh-python35/root/usr/lib64/python3.5/ctypes/test/__pycache__/test_simplesubclasses.cpython-35.opt-2.pyc 2015-12-06T20:39:07 /opt/rh/rh-python35/root/usr/lib64/python3.5/ctypes/test/test_simplesubclasses.py 2016-08-06T06:53:20
<snip>
#

Verified on all rhscl-2.3 packages:
# for file in *.rpm; do echo $file; rpmlint $file | grep python-bytecode-inconsistent-mtime; done
rh-python35-2.0-2.el6.x86_64.rpm
rh-python35-babel-1.3-2.el6.noarch.rpm
rh-python35-mod_wsgi-4.4.21-1.el6.x86_64.rpm
rh-python35-mod_wsgi-debuginfo-4.4.21-1.el6.x86_64.rpm
rh-python35-numpy-1.10.4-4.el6.x86_64.rpm
rh-python35-numpy-debuginfo-1.10.4-4.el6.x86_64.rpm
rh-python35-numpy-f2py-1.10.4-4.el6.x86_64.rpm
rh-python35-python-3.5.1-11.el6.x86_64.rpm
rh-python35-python-3.5.1-11.el7.x86_64.rpm
rh-python35-python-babel-1.3-2.el6.noarch.rpm
rh-python35-python-bson-3.2.1-2.el6.x86_64.rpm
rh-python35-python-coverage-4.0.3-1.el6.x86_64.rpm
rh-python35-python-coverage-debuginfo-4.0.3-1.el6.x86_64.rpm
rh-python35-python-debug-3.5.1-11.el6.x86_64.rpm
rh-python35-python-debug-3.5.1-11.el7.x86_64.rpm
rh-python35-python-debuginfo-3.5.1-11.el6.x86_64.rpm
rh-python35-python-debuginfo-3.5.1-11.el7.x86_64.rpm
rh-python35-python-devel-3.5.1-11.el6.x86_64.rpm
rh-python35-python-devel-3.5.1-11.el7.x86_64.rpm
rh-python35-python-docutils-0.12-2.el6.noarch.rpm
rh-python35-python-docutils-0.12-2.el7.noarch.rpm
rh-python35-python-jinja2-2.8-2.el6.noarch.rpm
rh-python35-python-libs-3.5.1-11.el6.x86_64.rpm
rh-python35-python-libs-3.5.1-11.el7.x86_64.rpm
rh-python35-python-markupsafe-0.23-1.el6.x86_64.rpm
rh-python35-python-markupsafe-debuginfo-0.23-1.el6.x86_64.rpm
rh-python35-python-nose-1.3.7-2.el6.noarch.rpm
rh-python35-python-nose-docs-1.3.7-2.el6.noarch.rpm
rh-python35-python-pip-7.1.0-2.el6.noarch.rpm
rh-python35-python-psycopg2-2.6.1-1.el6.x86_64.rpm
rh-python35-python-psycopg2-debuginfo-2.6.1-1.el6.x86_64.rpm
rh-python35-python-psycopg2-doc-2.6.1-1.el6.x86_64.rpm
rh-python35-python-pygments-2.1-1.el6.noarch.rpm
rh-python35-python-pymongo-3.2.1-2.el6.x86_64.rpm
rh-python35-python-pymongo-debuginfo-3.2.1-2.el6.x86_64.rpm
rh-python35-python-pymongo-doc-3.2.1-2.el6.x86_64.rpm
rh-python35-python-pymongo-gridfs-3.2.1-2.el6.x86_64.rpm
rh-python35-python-PyMySQL-0.7.5-1.el6.noarch.rpm
rh-python35-python-PyMySQL-0.7.5-1.el7.noarch.rpm
rh-python35-python-setuptools-18.0.1-2.el6.noarch.rpm
rh-python35-python-simplejson-3.6.5-1.el6.x86_64.rpm
rh-python35-python-simplejson-debuginfo-3.6.5-1.el6.x86_64.rpm
rh-python35-python-six-1.10.0-1.el6.noarch.rpm
rh-python35-python-sphinx-1.2.2-2.el6.noarch.rpm
rh-python35-python-sphinx-doc-1.2.2-2.el6.noarch.rpm
rh-python35-python-sqlalchemy-1.0.11-1.el6.x86_64.rpm
rh-python35-python-sqlalchemy-debuginfo-1.0.11-1.el6.x86_64.rpm
rh-python35-python-test-3.5.1-11.el6.x86_64.rpm
rh-python35-python-test-3.5.1-11.el7.x86_64.rpm
rh-python35-python-tkinter-3.5.1-11.el6.x86_64.rpm
rh-python35-python-tkinter-3.5.1-11.el7.x86_64.rpm
rh-python35-python-tools-3.5.1-11.el6.x86_64.rpm
rh-python35-python-tools-3.5.1-11.el7.x86_64.rpm
rh-python35-python-virtualenv-13.1.2-2.el6.noarch.rpm
rh-python35-python-virtualenv-13.1.2-2.el7.noarch.rpm
rh-python35-python-wheel-0.24.0-1.el6.noarch.rpm
rh-python35-PyYAML-3.10-14.el6.x86_64.rpm
rh-python35-PyYAML-debuginfo-3.10-14.el6.x86_64.rpm
rh-python35-runtime-2.0-2.el6.x86_64.rpm
rh-python35-scipy-0.17.0-2.el6.x86_64.rpm
rh-python35-scipy-debuginfo-0.17.0-2.el6.x86_64.rpm
rh-python35-scldevel-2.0-2.el6.x86_64.rpm
#
Comment 8 errata-xmlrpc 2016-11-15 10:00:24 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-2720.html
Note You need to log in before you can comment on or make changes to this bug.