(In reply to Miro Hrončok from comment #0)
...
> 
> A Python 3.8 change that I suspect:
> https://docs.python.org/dev/whatsnew/3.8.html#debug-build-uses-the-same-abi-
> as-release-build
> 
> It seems that _py3hbac.so is not linked against -lpython3.8.

Hi,

thank you for making us aware of this. If I understand the linked document correctly it is expected with python3.8 the C extensions do not link against -lpython3.8 to make them work with static build as well, is this correct? In this case I think you check tool for dynamically loaded objects has to be linked with -lpython3.8  to make the test pass again. I'll check.

bye,
Sumit

> it is expected with python3.8 the C extensions do not link against -lpython3.8

Correct.

> to make them work with static build as well, is this correct?

Correct. Static build and debug build.

> you check tool for dynamically loaded objects has to be linked with -lpython3.8

Not sure what this is but yes. Code that is not just a extension module needs to be linked has to be linked with -lpython3.8.

Also note that it was -lpython3.7m and now it is -lpython3.8 (without m).

Depending on how you get the flags, I can give you some hints to get this value dynamically. For example, `python3-config --abiflags` gives "" (empty string) on 3.8, and "m" on 3.7.

$ python3.8-config --abiflags

$ python3.7-config --abiflags
m

If you need something to test this in, this copr has some information on how to setup mock with Python 3.8:

https://copr.fedorainfracloud.org/coprs/g/python/python3.8/

We plan to start doing this in Fedora 31 in +- 1 week, so if we figure out how to at least workaround this before that, it would be great.

Upstream ticket:
https://pagure.io/SSSD/sssd/issue/4016

Hi,

pull-request https://github.com/SSSD/sssd/pull/819 should fix it.

bye,
Sumit

(In reply to Sumit Bose from comment #5)
> Hi,
> 
> pull-request https://github.com/SSSD/sssd/pull/819 should fix it.
> 
> bye,
> Sumit

Sumit,
It is not ideal solution.

Just _py3hbac.so, _py3sss.so, _py3sss_murmur.so, _py3sss_nss_idmap.so need aditional symbols.
There is not any reason why other files would need python3.8 symbols.

Solution should be the same as for other files which have undefined symbols provided by other binary.
e.g. winbind_idmap_sss.so

Moreover it should be checked just for python3.8+. Because we need to ensure there are not undefined symbols when linked with python3.7

(In reply to Lukas Slebodnik from comment #6)
> (In reply to Sumit Bose from comment #5)
> > Hi,
> > 
> > pull-request https://github.com/SSSD/sssd/pull/819 should fix it.
> > 
> > bye,
> > Sumit
> 
> Sumit,
> It is not ideal solution.
> 
> Just _py3hbac.so, _py3sss.so, _py3sss_murmur.so, _py3sss_nss_idmap.so need
> aditional symbols.
> There is not any reason why other files would need python3.8 symbols.
> 
> Solution should be the same as for other files which have undefined symbols
> provided by other binary.
> e.g. winbind_idmap_sss.so

Hi Lukas,

I was thinking about this as well but I decided to link libpython to make it easier to maintain this test. My understanding is that libpython is a general purpose library where is it expected that it is linked to all kind of program. This is e.g. different to libidmap-samba4.so which contains the missing symbol for winbind_idmap_sss.so where it is only expected that Samba executable will load it, it might be even possible that it is not available at all on some platforms. So instead of tracking all used symbols and manage them in a separate test library it looks more straight forward to me to directly use the library which should solve all dependencies.

What kind of drawbacks do you see?
 
> 
> Moreover it should be checked just for python3.8+. Because we need to ensure
> there are not undefined symbols when linked with python3.7

This should be handled by the configure check since the '--embed' option is only available in 3.8+ and will cause an error in older version.

bye,
Sumit

(In reply to Sumit Bose from comment #7)
> (In reply to Lukas Slebodnik from comment #6)
> > (In reply to Sumit Bose from comment #5)
> > > Hi,
> > > 
> > > pull-request https://github.com/SSSD/sssd/pull/819 should fix it.
> > > 
> > > bye,
> > > Sumit
> > 
> > Sumit,
> > It is not ideal solution.
> > 
> > Just _py3hbac.so, _py3sss.so, _py3sss_murmur.so, _py3sss_nss_idmap.so need
> > aditional symbols.
> > There is not any reason why other files would need python3.8 symbols.
> > 
> > Solution should be the same as for other files which have undefined symbols
> > provided by other binary.
> > e.g. winbind_idmap_sss.so
> 
> Hi Lukas,
> 
> I was thinking about this as well but I decided to link libpython to make it
> easier to maintain this test. My understanding is that libpython is a
> general purpose library where is it expected that it is linked to all kind
> of program. This is e.g. different to libidmap-samba4.so which contains the
> missing symbol for winbind_idmap_sss.so where it is only expected that Samba
> executable will load it, it might be even possible that it is not available
> at all on some platforms. So instead of tracking all used symbols and manage
> them in a separate test library it looks more straight forward to me to
> directly use the library which should solve all dependencies.
> 
> What kind of drawbacks do you see?
>  

libpython3.8.so provides lots of of public symbols. (sure most of them are prefixed with Py or _Py)
But not all of them. And you never know what will be provided in future.

sh-5.0# nm --dynamic --defined-only /usr/lib64/libpython3.8.so | wc -l
1960

And the dlopen should be super minimal to catch missing symbols. Libraries linked to
the dlopen-tests might cause false positive test.

e.g. 
So far dlopen test cannot detect missing linking to libdl
and it caused problems on debian for sssd version of libwbclient.so in past.

BTW linking dlopen-tests with libpython3 will not help if binary was compiled with linker flag --as-needed. Because unnecessary libraries will be filtered out and thus test will fail.

Can we at least get an immediate workaround and deal with a proper solution later? We would like to proceed with the rebuilds after beta1 is out and that should happen on Friday.

Changing severity to urgent because this is in critpath (according to PDC) and we'd like to proceed with 3.8 rebuilds next week.

This issue is blocking the Python 3.8 rebuilds. If this package won't build with 3.8, it won't be installable, along with all its dependent packages, after the side tag is merged.
Furthermore, as it fails to install, its dependent packages will fail to install and/or build as well. The fix should be pushed on the master branch and no release bump is required.

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.8: https://copr.fedorainfracloud.org/coprs/g/python/python3.8/

This issue needs to be resolved by next week, if other rebuilds of Python 3.8 beta 1 go well. If this is unrealistic for you, let us know how much time you need.

If you don't have free cycles to dedicate fixing your package, notify us and we'll try to provide some pointers. Let us know if we can push a fix directly without a pull request, in the case we happen to have one before you do.

We recommend always consulting with upstream, chances are this is already fixed there.

* master: 80fdef5
* sssd-1-16: 8f45a02

Michal, please package..

This issue is blocking the Python 3.8 rebuilds. This package is critical for Fedora, unless this is resolved, we cannot proceed with the side tag merge.

I am working on this, it will probably not be done today, but left for tomorrow due to some rawhide specific sssd build failures.

It was trivial to fix issue on rawhide after investiagation of failed build.log

Here you are:
diff --git a/sssd.spec b/sssd.spec
index 74c36ad..6bf4eaf 100644
--- a/sssd.spec
+++ b/sssd.spec
@@ -539,6 +539,8 @@ autoreconf -ivf
 
 make %{?_smp_mflags} all docs
 
+sed -i -e 's:/usr/bin/python:/usr/bin/python3:' src/tools/sss_obfuscate
+
 %check
 export CK_TIMEOUT_MULTIPLIER=10
 make %{?_smp_mflags} check VERBOSE=yes
@@ -546,8 +548,6 @@ unset CK_TIMEOUT_MULTIPLIER
 
 %install
 
-sed -i -e 's:/usr/bin/python:/usr/bin/python3:' src/tools/sss_obfuscate
-
 make install DESTDIR=$RPM_BUILD_ROOT
 
 if [ ! -f $RPM_BUILD_ROOT/%{_libdir}/%{name}/modules/libwbclient.so.%{libwbc_lib_version} ]

(In reply to Lukas Slebodnik from comment #17)
> It was trivial to fix issue on rawhide after investiagation of failed
> build.log
> 
> Here you are:
> diff --git a/sssd.spec b/sssd.spec
> index 74c36ad..6bf4eaf 100644
> --- a/sssd.spec
> +++ b/sssd.spec
> @@ -539,6 +539,8 @@ autoreconf -ivf
>  
>  make %{?_smp_mflags} all docs
>  
> +sed -i -e 's:/usr/bin/python:/usr/bin/python3:' src/tools/sss_obfuscate
> +
>  %check
>  export CK_TIMEOUT_MULTIPLIER=10
>  make %{?_smp_mflags} check VERBOSE=yes
> @@ -546,8 +548,6 @@ unset CK_TIMEOUT_MULTIPLIER
>  
>  %install
>  
> -sed -i -e 's:/usr/bin/python:/usr/bin/python3:' src/tools/sss_obfuscate
> -
>  make install DESTDIR=$RPM_BUILD_ROOT
>  
>  if [ ! -f
> $RPM_BUILD_ROOT/%{_libdir}/%{name}/modules/libwbclient.so.
> %{libwbc_lib_version} ]

Thanks.

Closing the issue. Should be fixed now in rawhide. Please reopen if the issue persists.

Verified. Thanks.