Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1968999 - F35FailsToInstall: python3-healpy; fails to build: invalid runpath '/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib'
Summary: F35FailsToInstall: python3-healpy; fails to build: invalid runpath '/builddir...
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: Fedora
Classification: Fedora
Component: python-healpy
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Christian Dersch
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On: 1936839
Blocks: PYTHON3.10 F35FTBFS, RAWHIDEFTBFS F35FailsToInstall, RAWHIDEFailsToInstall 1968899 1969093
TreeView+ depends on / blocked
 
Reported: 2021-06-07 23:40 UTC by Miro Hrončok
Modified: 2021-07-12 11:19 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-07-05 22:07:28 UTC
Type: ---
Embargoed:

Attachments (Terms of Use)

Description Miro Hrončok 2021-06-07 23:40:01 UTC 
Hello,

Please note that this comment was generated automatically. If you feel that this output has mistakes, please contact me via email (mhroncok).

Your package (python-healpy) Fails To Install in Fedora 35:

can't install python3-healpy:
  - nothing provides python3.9dist(matplotlib) needed by python3-healpy-1.14.0-4.fc34.x86_64
  - nothing provides python3.9dist(scipy) needed by python3-healpy-1.14.0-4.fc34.x86_64
  - nothing provides python3.9dist(six) needed by python3-healpy-1.14.0-4.fc34.x86_64
  - nothing provides python(abi) = 3.9 needed by python3-healpy-1.14.0-4.fc34.x86_64
  - nothing provides python3.9dist(numpy) >= 1.13 needed by python3-healpy-1.14.0-4.fc34.x86_64
  
If you know about this problem and are planning on fixing it, please acknowledge so by setting the bug status to ASSIGNED. If you don't have time to maintain this package, consider orphaning it, so maintainers of dependent packages realize the problem.


If you don't react accordingly to the policy for FTBFS/FTI bugs (https://docs.fedoraproject.org/en-US/fesco/Fails_to_build_from_source_Fails_to_install/), your package may be orphaned in 8+ weeks.

P.S. The data was generated solely from koji buildroot, so it might be newer than the latest compose or the content on mirrors.

P.P.S. If this bug has been reported in the middle of upgrading multiple dependent packages, please consider using side tags: https://docs.fedoraproject.org/en-US/fesco/Updates_Policy/#updating-inter-dependent-packages

Thanks!
Comment 1 Miro Hrončok 2021-06-07 23:46:01 UTC 
This bugzilla is likely a fallout from the Python 3.10 rebuild.

If your package (or some of the dependencies it has) failed to rebuild during the Python 3.10 rebuild, they now fail to install. To fix this, packages need to be rebuilt in Rawhide.

We will slowly triage the bugzillas, but we'd appreciate your help.

If you know this is blocked by an existing reported build failure or another package not yet rebuilt with Python 3.10, please mark it as such by using the "Depends On"/"Blocks" bugzilla fields. That will help us determine what failures to prioritize.

Thank you and sorry for the inconvenience. Let me know if you need any help.
Comment 2 Miro Hrončok 2021-06-15 20:50:35 UTC 
Hello,

This is the first reminder (step 3 from https://docs.fedoraproject.org/en-US/fesco/Fails_to_build_from_source_Fails_to_install/#_package_removal_for_long_standing_ftbfs_and_fti_bugs).

If you know about this problem and are planning on fixing it, please acknowledge so by setting the bug status to ASSIGNED. If you don't have time to maintain this package, consider orphaning it, so maintainers of dependent packages realize the problem.
Comment 3 Zbigniew Jędrzejewski-Szmek 2021-06-17 20:21:26 UTC 
+ /usr/lib/rpm/check-rpaths
*******************************************************************************
*
* WARNING: 'check-rpaths' detected a broken RPATH OR RUNPATH and will cause
*          'rpmbuild' to fail. To ignore these errors, you can set the
*          '$QA_RPATHS' environment variable which is a bitmask allowing the
*          values below. The current value of QA_RPATHS is 0x0000.
*
*    0x0001 ... standard RPATHs (e.g. /usr/lib); such RPATHs are a minor
*               issue but are introducing redundant searchpaths without
*               providing a benefit. They can also cause errors in multilib
*               environments.
*    0x0002 ... invalid RPATHs; these are RPATHs which are neither absolute
*               nor relative filenames and can therefore be a SECURITY risk
*    0x0004 ... insecure RPATHs; these are relative RPATHs which are a
*               SECURITY risk
*    0x0008 ... the special '$ORIGIN' RPATHs are appearing after other
*               RPATHs; this is just a minor issue but usually unwanted
*    0x0010 ... the RPATH is empty; there is no reason for such RPATHs
*               and they cause unneeded work while loading libraries
*    0x0020 ... an RPATH references '..' of an absolute path; this will break
*               the functionality when the path before '..' is a symlink
*          
*
* Examples:
* - to ignore standard and empty RPATHs, execute 'rpmbuild' like
*   $ QA_RPATHS=$(( 0x0001|0x0010 )) rpmbuild my-package.src.rpm
* - to check existing files, set $RPM_BUILD_ROOT and execute check-rpaths like
*   $ RPM_BUILD_ROOT=<top-dir> /usr/lib/rpm/check-rpaths
*  
*******************************************************************************
ERROR   0002: file '/usr/lib64/python3.10/site-packages/healpy/_healpy_pixel_lib.cpython-310-x86_64-linux-gnu.so' contains an invalid runpath '/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib' in [/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib]
ERROR   0002: file '/usr/lib64/python3.10/site-packages/healpy/_healpy_sph_transform_lib.cpython-310-x86_64-linux-gnu.so' contains an invalid runpath '/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib' in [/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib]
ERROR   0002: file '/usr/lib64/python3.10/site-packages/healpy/_query_disc.cpython-310-x86_64-linux-gnu.so' contains an invalid runpath '/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib' in [/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib]
ERROR   0002: file '/usr/lib64/python3.10/site-packages/healpy/_sphtools.cpython-310-x86_64-linux-gnu.so' contains an invalid runpath '/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib' in [/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib]
ERROR   0002: file '/usr/lib64/python3.10/site-packages/healpy/_pixelfunc.cpython-310-x86_64-linux-gnu.so' contains an invalid runpath '/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib' in [/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib]
ERROR   0002: file '/usr/lib64/python3.10/site-packages/healpy/_masktools.cpython-310-x86_64-linux-gnu.so' contains an invalid runpath '/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib' in [/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib]
ERROR   0002: file '/usr/lib64/python3.10/site-packages/healpy/_hotspots.cpython-310-x86_64-linux-gnu.so' contains an invalid runpath '/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib' in [/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib]
ERROR   0002: file '/usr/lib64/python3.10/site-packages/healpy/_line_integral_convolution.cpython-310-x86_64-linux-gnu.so' contains an invalid runpath '/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib' in [/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib]
error: Bad exit status from /var/tmp/rpm-tmp.tYjDsc (%install)
    Bad exit status from /var/tmp/rpm-tmp.tYjDsc (%install)
Comment 4 Mamoru TASAKA 2021-06-23 12:20:48 UTC 
Currently the FailsToInstall issue of this package causes some looong dependency chain breakage, and it results in Astronomy Live spin creation failure: https://koji.fedoraproject.org/koji/taskinfo?taskID=70671815

While I am not the maintainer of this package, as I want to keep Astronomy Live spin healthy, I want to make this package fixed ASAP.
Currently I have not figured out how to remove invalid rpath "in reasonable way" - if I cannot find out in a few days,
I am going to resort to using "chrpath -r" method and push the tentative fix and build this package.
Comment 5 Miro Hrončok 2021-06-23 13:12:41 UTC 
> Currently I have not figured out how to remove invalid rpath "in reasonable way"

Me neither. I've tried sedding LD_RUN_PATH to DIE_RPATH_DIE in confugre scripts, I've tried adding --disable-rpath to setup.py's configure invocation. No dice.
Comment 6 Mattia Verga 2021-06-26 09:33:26 UTC 
I tried to do a scratch build of healpy 1.15.0, but it fails with the same error.

I think the problem is due to the bundled libsharp library:

checking if library 'sharp' is installed
Package libsharp was not found in the pkg-config search path.
Perhaps you should add the directory containing `libsharp.pc'
to the PKG_CONFIG_PATH environment variable
Package 'libsharp', required by 'virtual:world', not found
building library 'sharp' from source
creating /builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10
creating /builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/sharp
/bin/sh /builddir/build/BUILD/healpy-1.14.0/healpixsubmodule/src/common_libraries/libsharp/configure --prefix=/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10 --disable-shared --with-pic --disable-maintainer-mode

and then:

make[1]: Entering directory '/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/sharp'
 /usr/bin/mkdir -p '/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib'
 /bin/sh ./libtool   --mode=install /usr/bin/install -c   libsharp.la '/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib'
libtool: install: /usr/bin/install -c .libs/libsharp.lai /builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib/libsharp.la
libtool: install: /usr/bin/install -c .libs/libsharp.a /builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib/libsharp.a
libtool: install: chmod 644 /builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib/libsharp.a
libtool: install: ranlib /builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib/libsharp.a
libtool: finish: PATH="/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/sbin:/sbin" ldconfig -n /builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib
----------------------------------------------------------------------
Libraries have been installed in:
   /builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib
If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the '-LLIBDIR'
flag during linking and do at least one of the following:
   - add LIBDIR to the 'LD_LIBRARY_PATH' environment variable
     during execution
   - add LIBDIR to the 'LD_RUN_PATH' environment variable
     during linking
   - use the '-Wl,-rpath -Wl,LIBDIR' linker flag
   - have your system administrator add LIBDIR to '/etc/ld.so.conf'
See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------
 /usr/bin/mkdir -p '/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/include'
 /usr/bin/mkdir -p '/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/include/libsharp'
 /usr/bin/install -c -m 644  /builddir/build/BUILD/healpy-1.14.0/healpixsubmodule/src/common_libraries/libsharp/libsharp/sharp.h /builddir/build/BUILD/healpy-1.14.0/healpixsubmodule/src/common_libraries/libsharp/libsharp/sharp_geomhelpers.h /builddir/build/BUILD/healpy-1.14.0/healpixsubmodule/src/common_libraries/libsharp/libsharp/sharp_almhelpers.h /builddir/build/BUILD/healpy-1.14.0/healpixsubmodule/src/common_libraries/libsharp/libsharp/sharp_cxx.h '/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/include/libsharp'
 /usr/bin/mkdir -p '/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib/pkgconfig'
 /usr/bin/install -c -m 644 libsharp.pc '/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib/pkgconfig'

I think the problem can be solved by unbundling libsharp, but I don't have enough skills for a quick fix while we package that library in Fedora.
Comment 7 Mattia Verga 2021-06-26 09:36:11 UTC 
oh, also libhealpix_cxx seems to be linked in the same way
Comment 8 Mamoru TASAKA 2021-07-04 07:07:59 UTC 
Unfortunately, repoquery shows that "pkgconfig(libsharp)" is not provided by any package...
Comment 9 Mamoru TASAKA 2021-07-04 07:24:44 UTC 
Modified:
https://src.fedoraproject.org/rpms/python-healpy/c/7c2b9915d36706ed029a8bec9b144d6fe57bd07d?branch=rawhide
https://koji.fedoraproject.org/koji/buildinfo?buildID=1777784

So the broken build sets invalid rpath by:
=======================================================
g++ -shared -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -g -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -g -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection build/temp.linux-x86_64-3.10/healpy/src/_healpy_pixel_lib.o -L/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib -L/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib -L/usr/lib64 -Lbuild/temp.linux-x86_64-3.10 -Wl,--enable-new-dtags,-R/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib -Wl,--enable-new-dtags,-R/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib -lcfitsio -lsharp -lm -lhealpix_cxx -lcfitsio -lsharp -lhealpix_cxx -o build/lib.linux-x86_64-3.10/healpy/_healpy_pixel_lib.cpython-310-x86_64-linux-gnu.so -D_REENTRANT -fopenmp -D_REENTRANT
=======================================================

Note that "-Wl,--enable-new-dtags,-R/builddir/build/BUILD/healpy-1.14.0/build/temp.linux-x86_64-3.10/lib".
"grep"ing "--enable-new-dtags" in buildroot hits:

$ grep -rnI enable-new-dtags ./usr/lib
./usr/lib/python3.10/site-packages/setuptools/_distutils/unixccompiler.py:253:                    return "-Wl,--enable-new-dtags,-R" + dir

Then looking at unixccompiler.py:
---------------------------------------------------
   220      def runtime_library_dir_option(self, dir):
   235          if sys.platform[:6] == "darwin":
   244          else:
   245              if self._is_gcc(compiler):
   250                  if sysconfig.get_config_var("GNULD") == "yes":
   251                      # GNU ld needs an extra option to get a RUNPATH
   252                      # instead of just an RPATH.
   253                      return "-Wl,--enable-new-dtags,-R" + dir
   254                  else:
   255                      return "-Wl,-R" + dir
---------------------------------------------------
and _distutils/ccompiler.py :
---------------------------------------------------
  1087  def gen_lib_options (compiler, library_dirs, runtime_library_dirs, libraries):
  1094      lib_opts = []
  1095  
  1096      for dir in library_dirs:
  1097          lib_opts.append(compiler.library_dir_option(dir))
  1098  
  1099      for dir in runtime_library_dirs:
  1100          opt = compiler.runtime_library_dir_option(dir)
  1101          if isinstance(opt, list):
  1102              lib_opts = lib_opts + opt
  1103          else:
  1104              lib_opts.append(opt)
---------------------------------------------------

So it seems that python3-setuptools adds "-Wl,--enable-new-dtags,-R" option using "runtime_library_dirs" environ.
Comment 10 Miro Hrončok 2021-07-04 08:14:30 UTC 
Note that setuptools/_distutils is likely never actually used. However standard library distutils probably does the same.
Comment 11 Miro Hrončok 2021-07-05 22:07:28 UTC 
Hello,

Please note that this comment was generated automatically. If you feel that this output has mistakes, please contact me via email (mhroncok).

All subpackages of a package against which this bug was filled are now installable or removed from Fedora 35.

Thanks for taking care of it!
Comment 12 Petr Viktorin 2021-07-12 11:15:39 UTC 
I've seen this in packages that use numpy.distutils, which monkeypatches distutils/setuptools and uses some rather convoluted logic around libraries.
Specifically, see https://github.com/numpy/numpy/blob/c07080f12652b2cced2f6ca2cb3e72e085565047/numpy/distutils/system_info.py#L741 -- I haven't figured out why library_dirs is copied to runtime_library_dirs, but it seems this is part of the path that adds rpath.
Comment 13 Petr Viktorin 2021-07-12 11:19:15 UTC 
thinko: numpy.distutils doesn't monkeypatch distutils but *extends* it.
Note You need to log in before you can comment on or make changes to this bug.