Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1959088 - petsc fails to build with Python 3.10: RuntimeError('PETSc makefiles contain mistakes or files are missing on filesystem...
Summary: petsc fails to build with Python 3.10: RuntimeError('PETSc makefiles contain ...
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: petsc
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Antonio T. sagitter
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1968855 (view as bug list)
Depends On:
Blocks: PYTHON3.10 F35FTBFS, RAWHIDEFTBFS F35FailsToInstall, RAWHIDEFailsToInstall
TreeView+ depends on / blocked
 
Reported: 2021-05-10 16:59 UTC by Miro Hrončok
Modified: 2021-06-11 08:15 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-06-11 08:15:02 UTC
Type: Bug
Embargoed:

Attachments (Terms of Use)
gmakegen.py (16.86 KB, text/x-python)
2021-05-11 18:41 UTC, Antonio T. sagitter 		no flags Details
View All


Links
System ID Private Priority Status Summary Last Updated
Python 44351 0 None None None 2021-06-08 19:00:22 UTC

Description Miro Hrončok 2021-05-10 16:59:46 UTC 
petsc fails to build with Python 3.10.0b1:

+ make V=1 MAKE_NP=2 PETSC_DIR=/builddir/build/BUILD/petsc-3.15.0/petsc-3.15.0 PETSC_ARCH=x86_64 all
/usr/bin/python3 ./config/gmakegen.py --petsc-arch=x86_64
/builddir/build/BUILD/petsc-3.15.0/petsc-3.15.0/./config/gmakegen.py:4: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
  from distutils.sysconfig import parse_makefile
/builddir/build/BUILD/petsc-3.15.0/petsc-3.15.0/./config/gmakegen.py:4: DeprecationWarning: The distutils.sysconfig module is deprecated, use sysconfig instead
  from distutils.sysconfig import parse_makefile
Traceback (most recent call last):
  File "/builddir/build/BUILD/petsc-3.15.0/petsc-3.15.0/./config/gmakegen.py", line 345, in <module>
    main(petsc_arch=opts.petsc_arch, pkg_dir=opts.pkg_dir, pkg_name=opts.pkg_name, pkg_arch=opts.pkg_arch, pkg_pkgs=opts.pkg_pkgs, output=opts.output, verbose=opts.verbose)
  File "/builddir/build/BUILD/petsc-3.15.0/petsc-3.15.0/./config/gmakegen.py", line 328, in main
    petsc.summary()
  File "/builddir/build/BUILD/petsc-3.15.0/petsc-3.15.0/./config/gmakegen.py", line 255, in summary
    self.mistakes.summary()
  File "/builddir/build/BUILD/petsc-3.15.0/petsc-3.15.0/./config/gmakegen.py", line 79, in summary
    raise RuntimeError('PETSc makefiles contain mistakes or files are missing on filesystem.\n%s\nPossible reasons:\n\t1. Files were deleted locally, try "hg revert filename" or "git checkout filename".\n\t2. Files were deleted from repository, but were not removed from makefile. Send mail to petsc-maint.gov.\n\t3. Someone forgot to "add" new files to the repository. Send mail to petsc-maint.gov.' % ('\n'.join(self.mistakes)))
RuntimeError: PETSc makefiles contain mistakes or files are missing on filesystem.
Makefile contains directory not on filesystem: /builddir/build/BUILD/petsc-3.15.0/petsc-3.15.0/src/sys: ['\\']
Makefile contains directory not on filesystem: /builddir/build/BUILD/petsc-3.15.0/petsc-3.15.0/src/mat/impls/aij/seq: ['\\']
Makefile contains directory not on filesystem: /builddir/build/BUILD/petsc-3.15.0/petsc-3.15.0/src/ksp/ksp/impls: ['\\']
Makefile contains directory not on filesystem: /builddir/build/BUILD/petsc-3.15.0/petsc-3.15.0/src/ksp/pc/impls: ['ml\\']
Makefile contains directory not on filesystem: /builddir/build/BUILD/petsc-3.15.0/petsc-3.15.0/src/tao/complementarity/impls: ['#', 'isls', 'rscs']
Possible reasons:
	1. Files were deleted locally, try "hg revert filename" or "git checkout filename".
	2. Files were deleted from repository, but were not removed from makefile. Send mail to petsc-maint.gov.
	3. Someone forgot to "add" new files to the repository. Send mail to petsc-maint.gov.
gmakefile:67: x86_64/lib/petsc/conf/files: No such file or directory

For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.10/fedora-rawhide-x86_64/02175460-petsc/

For all our attempts to build petsc with Python 3.10, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.10/package/petsc/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.10:
https://copr.fedorainfracloud.org/coprs/g/python/python3.10/

Let us know here if you have any questions.

Python 3.10 will be included in Fedora 35. To make that update smoother, we're building Fedora packages with early pre-releases of Python 3.10.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.
Comment 1 Antonio T. sagitter 2021-05-11 17:10:46 UTC 
Hi Miro.

How is imported `parse_makefile` with Python-3.10?
Comment 2 Miro Hrončok 2021-05-11 17:46:52 UTC 
I am sorry, but I don't understand your question.
Comment 3 Antonio T. sagitter 2021-05-11 18:41:33 UTC 
Created attachment 1782125 [details]
gmakegen.py

petsc is failing with:

+ make V=1 MAKE_NP=2 PETSC_DIR=/builddir/build/BUILD/petsc-3.15.0/petsc-3.15.0 PETSC_ARCH=x86_64 all
/usr/bin/python3 ./config/gmakegen.py --petsc-arch=x86_64
Traceback (most recent call last):
  File "/builddir/build/BUILD/petsc-3.15.0/petsc-3.15.0/./config/gmakegen.py", line 4, in <module>
    from sysconfig import parse_makefile
ImportError: cannot import name 'parse_makefile' from 'sysconfig' (/usr/lib64/python3.10/sysconfig.py)
gmakefile:67: x86_64/lib/petsc/conf/files: No such file or directory
Comment 4 Miro Hrončok 2021-05-11 22:43:49 UTC 
Line 4 is actually "from distutils.sysconfig import parse_makefile"



[~]$ python3.9
Python 3.9.5 (default, May  4 2021, 00:00:00) 
[GCC 10.3.1 20210422 (Red Hat 10.3.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from distutils.sysconfig import parse_makefile
>>> 

[~]$ python3.10
Python 3.10.0b1 (default, May  4 2021, 00:00:00) [GCC 10.3.1 20210422 (Red Hat 10.3.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from distutils.sysconfig import parse_makefile
<stdin>:1: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
<stdin>:1: DeprecationWarning: The distutils.sysconfig module is deprecated, use sysconfig instead
>>> 


Works in both versions, but is deprecated. Not sure where "from sysconfig import parse_makefile" comes from.
Comment 5 Antonio T. sagitter 2021-05-31 18:23:06 UTC 
I don't know how to fix this error.
Please, take a look.
Comment 6 Antonio T. sagitter 2021-05-31 18:25:48 UTC 
(In reply to Antonio T. sagitter from comment #5)
> I don't know how to fix this error.
> Please, take a look.

I refer to https://bugzilla.redhat.com/show_bug.cgi?id=1959088#c0
Comment 7 Miro Hrončok 2021-06-04 20:14:10 UTC 
This is a mass-posted update. Sorry if it is not 100% accurate to this bugzilla.


The Python 3.10 rebuild is in progress in a Koji side tag. If you manage to fix the problem, please commit the fix in the rawhide branch, but don't build the package in regular rawhide.

You can either build the package in the side tag, with:

    $ fedpkg build --target=f35-python

Or you can the build and we will eventually build it for you.

Note that the rebuild is still in progress, so not all (build) dependencies of this package might be available right away.

Thanks.

See also https://fedoraproject.org/wiki/Changes/Python3.10

If you have general questions about the rebuild, please use this mailing list thread: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/G47SGOYIQLRDTWGOSLSWERZSSHXDEDH5/
Comment 8 Miro Hrončok 2021-06-07 22:58:51 UTC 
The f35-python side tag has been merged to Rawhide. From now on, build as you would normally build.
Comment 9 Miro Hrončok 2021-06-08 11:19:17 UTC 
*** Bug 1968855 has been marked as a duplicate of this bug. ***
Comment 10 Miro Hrončok 2021-06-08 18:39:45 UTC 
I've set PYTHONWARNINGS=ignore::DeprecationWarning to verify the DeprecationWarning is not the problem.


+ make V=1 MAKE_NP=8 PETSC_DIR=/builddir/build/BUILD/petsc-3.15.0/petsc-3.15.0 PETSC_ARCH=x86_64 all
/usr/bin/python3 ./config/gmakegen.py --petsc-arch=x86_64
Traceback (most recent call last):
  File "/builddir/build/BUILD/petsc-3.15.0/petsc-3.15.0/./config/gmakegen.py", line 345, in <module>
    main(petsc_arch=opts.petsc_arch, pkg_dir=opts.pkg_dir, pkg_name=opts.pkg_name, pkg_arch=opts.pkg_arch, pkg_pkgs=opts.pkg_pkgs, output=opts.output, verbose=opts.verbose)
  File "/builddir/build/BUILD/petsc-3.15.0/petsc-3.15.0/./config/gmakegen.py", line 328, in main
    petsc.summary()
  File "/builddir/build/BUILD/petsc-3.15.0/petsc-3.15.0/./config/gmakegen.py", line 255, in summary
    self.mistakes.summary()
  File "/builddir/build/BUILD/petsc-3.15.0/petsc-3.15.0/./config/gmakegen.py", line 79, in summary
    raise RuntimeError('PETSc makefiles contain mistakes or files are missing on filesystem.\n%s\nPossible reasons:\n\t1. Files were deleted locally, try "hg revert filename" or "git checkout filename".\n\t2. Files were deleted from repository, but were not removed from makefile. Send mail to petsc-maint.gov.\n\t3. Someone forgot to "add" new files to the repository. Send mail to petsc-maint.gov.' % ('\n'.join(self.mistakes)))
RuntimeError: PETSc makefiles contain mistakes or files are missing on filesystem.
Makefile contains directory not on filesystem: /builddir/build/BUILD/petsc-3.15.0/petsc-3.15.0/src/sys: ['\\']
Makefile contains directory not on filesystem: /builddir/build/BUILD/petsc-3.15.0/petsc-3.15.0/src/mat/impls/aij/seq: ['\\']
Makefile contains directory not on filesystem: /builddir/build/BUILD/petsc-3.15.0/petsc-3.15.0/src/ksp/pc/impls: ['ml\\']
Makefile contains directory not on filesystem: /builddir/build/BUILD/petsc-3.15.0/petsc-3.15.0/src/ksp/ksp/impls: ['\\']
Makefile contains directory not on filesystem: /builddir/build/BUILD/petsc-3.15.0/petsc-3.15.0/src/tao/complementarity/impls: ['#', 'isls', 'rscs']
Possible reasons:
	1. Files were deleted locally, try "hg revert filename" or "git checkout filename".
	2. Files were deleted from repository, but were not removed from makefile. Send mail to petsc-maint.gov.
	3. Someone forgot to "add" new files to the repository. Send mail to petsc-maint.gov.
gmakefile:67: x86_64/lib/petsc/conf/files: No such file or directory
make: *** [gmakefile:63: x86_64/lib/petsc/conf/files] Error 1
make: *** Deleting file 'x86_64/lib/petsc/conf/files'





And I found the cause:

[petsc]$ cd petsc-3.15.0/petsc-3.15.0/src/ksp/pc/impls
[impls]$ python3.9
Python 3.9.5 (default, May 14 2021, 00:00:00) 
[GCC 10.3.1 20210422 (Red Hat 10.3.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from distutils.sysconfig import parse_makefile
>>> makevars = parse_makefile('makefile')
>>> makevars.get('DIRS','').split()
['jacobi', 'none', 'sor', 'shell', 'bjacobi', 'mg', 'eisens', 'asm', 'ksp', 'composite', 'redundant', 'spai', 'is', 'pbjacobi', 'vpbjacobi', 'ml', 'mat', 'hypre', 'tfs', 'fieldsplit', 'factor', 'galerkin', 'cp', 'wb', 'python', 'chowiluviennacl', 'chowiluviennaclcuda', 'rowscalingviennacl', 'rowscalingviennaclcuda', 'saviennacl', 'saviennaclcuda', 'lsc', 'redistribute', 'gasm', 'svd', 'gamg', 'parms', 'bddc', 'kaczmarz', 'telescope', 'patch', 'lmvm', 'hmg', 'deflation', 'hpddm', 'hara']


[impls]$ python3.10
Python 3.10.0b2 (default, Jun  1 2021, 00:00:00) [GCC 10.3.1 20210422 (Red Hat 10.3.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from distutils.sysconfig import parse_makefile
<stdin>:1: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
<stdin>:1: DeprecationWarning: The distutils.sysconfig module is deprecated, use sysconfig instead
>>> makevars = parse_makefile('makefile')
>>> makevars.get('DIRS','').split()
['jacobi', 'none', 'sor', 'shell', 'bjacobi', 'mg', 'eisens', 'asm', 'ksp', 'composite', 'redundant', 'spai', 'is', 'pbjacobi', 'vpbjacobi', 'ml\\']




[petsc]$ cd petsc-3.15.0/petsc-3.15.0/src/ksp/ksp/impls
[impls]$ python3.9
Python 3.9.5 (default, May 14 2021, 00:00:00) 
[GCC 10.3.1 20210422 (Red Hat 10.3.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from distutils.sysconfig import parse_makefile
>>> makevars = parse_makefile('makefile')
>>> makevars.get('DIRS','').split()
['cr', 'bcgs', 'bcgsl', 'cg', 'cgs', 'gmres', 'cheby', 'rich', 'lsqr', 'preonly', 'tcqmr', 'tfqmr', 'qcg', 'bicg', 'minres', 'symmlq', 'lcd', 'ibcgs', 'python', 'gcr', 'fcg', 'tsirm', 'fetidp', 'hpddm']


[impls]$ python3.10
Python 3.10.0b2 (default, Jun  1 2021, 00:00:00) [GCC 10.3.1 20210422 (Red Hat 10.3.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from distutils.sysconfig import parse_makefile
<stdin>:1: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
<stdin>:1: DeprecationWarning: The distutils.sysconfig module is deprecated, use sysconfig instead
>>> makevars = parse_makefile('makefile')
>>> makevars.get('DIRS','').split()
['cr', 'bcgs', 'bcgsl', 'cg', 'cgs', 'gmres', 'cheby', 'rich', 'lsqr', 'preonly', 'tcqmr', 'tfqmr', '\\']





[petsc]$ cd petsc-3.15.0/petsc-3.15.0/src/tao/complementarity/impls
[impls]$ python3.9
Python 3.9.5 (default, May 14 2021, 00:00:00) 
[GCC 10.3.1 20210422 (Red Hat 10.3.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from distutils.sysconfig import parse_makefile
>>> makevars = parse_makefile('makefile')
>>> makevars.get('DIRS','').split()
['ssls', 'asls']


[impls]$ python3.10
Python 3.10.0b2 (default, Jun  1 2021, 00:00:00) [GCC 10.3.1 20210422 (Red Hat 10.3.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from distutils.sysconfig import parse_makefile
<stdin>:1: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
<stdin>:1: DeprecationWarning: The distutils.sysconfig module is deprecated, use sysconfig instead
>>> makevars = parse_makefile('makefile')
>>> makevars.get('DIRS','').split()
['ssls', 'asls', '#', 'isls', 'rscs']





parse_makefile chnaged behavior. I'll report that to Python upstream.
Comment 11 Miro Hrončok 2021-06-08 19:00:23 UTC 
Reported as https://bugs.python.org/issue44351
Comment 12 Miro Hrončok 2021-06-08 22:57:14 UTC 
I hope this will fix the build in the meantime: https://src.fedoraproject.org/rpms/petsc/pull-request/4
Comment 13 Antonio T. sagitter 2021-06-11 08:15:02 UTC 
> I hope this will fix the build in the meantime:
> https://src.fedoraproject.org/rpms/petsc/pull-request/4

Fixed.
Thank you(In reply to Miro Hrončok from comment #12)
Note You need to log in before you can comment on or make changes to this bug.