Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1193839 - sssd loses sudo rules on boot
Summary: sssd loses sudo rules on boot
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: sssd
Version: 22
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Jakub Hrozek
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-02-18 11:42 UTC by Fabrice Robin
Modified: 2020-05-02 18:05 UTC (History)
8 users (show)

Fixed In Version: sssd-1.13.3-1.fc23 sssd-1.13.3-1.fc22
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-12-19 18:25:31 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
sssd logs (42.67 KB, application/zip)
2015-02-18 15:06 UTC, Fabrice Robin
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Github SSSD sssd issues 3713 0 None None None 2020-05-02 18:05:43 UTC

Description Fabrice Robin 2015-02-18 11:42:28 UTC
Description of problem:
We are using sudoHost ldap attributes with address/netmask form: xx.y.zz.0/24
And the rules defined with this attribute are seldom functionals for our users.

We have to stop the sssd service, delete the sssd folder /var/lib/sssd/db, restart the sssd service, in order to have those rules functionals again with sudo.
But the sudo rules are always lost on next reboot...

(The rules defined with sudoHost=ALL are always functionals !).


Version-Release number of selected component (if applicable):
sssd.x86_64                                          1.12.3-4.fc21
sssd-ad.x86_64                                       1.12.3-4.fc21
sssd-client.x86_64                                   1.12.3-4.fc21
sssd-common.x86_64                                   1.12.3-4.fc21
sssd-common-pac.x86_64                               1.12.3-4.fc21
sssd-ipa.x86_64                                      1.12.3-4.fc21
sssd-krb5.x86_64                                     1.12.3-4.fc21
sssd-krb5-common.x86_64                              1.12.3-4.fc21
sssd-ldap.x86_64                                     1.12.3-4.fc21
sssd-proxy.x86_64                                    1.12.3-4.fc21


How reproducible:
Always

Steps to Reproduce:
1. Define a sudoRole with a sudoHost attribute as xx.y.zz.0/24 and sudoUser as aaa
2. Start a fedora machine on the network xx.y.zz.0/24
3. This machine sssd.conf is defined as in "Additional info"
4. Login as aaa on the machine
5. Execute a sudo command

Actual results: 
aaa is not allowed to run sudo on xxx. This incident will be reported.

Expected results:
The user should be able to execute the sudo command

Additional info:
We tried to use ldap_sudo_smart_refresh_interval and ldap_sudo_full_refresh_interval with very small intervals without success...

The only solution we know is to stop the sssd service, delete /var/lib/sssd/db, and restart the service.
We are aware of this bug for years now.

sssd.conf =>
ldap_uri = _srv_,ldap://ldap.xxx.xxx
ldap_chpass_uri = ldap://ldap.xxx.xxx
ldap_tls_reqcert = allow
enumerate = false
ldap_sudo_search_base = ou=SUDOers,dc=xxx,dc=xxx

autofs_provider = ldap
auth_provider = ldap
krb5_realm = #
ldap_search_base = dc=xxx,dc=xxx
id_provider = ldap
ldap_id_use_start_tls = True
chpass_provider = ldap
cache_credentials = True
ldap_tls_cacertdir = /etc/openldap/cacerts
[sssd]
services = nss, pam, sudo, autofs
config_file_version = 2

domains = default
[nss]
homedir_substring = /home

[pam]

[sudo]

[autofs]

[ssh]

Comment 1 Lukas Slebodnik 2015-02-18 11:56:58 UTC
(In reply to Fabrice Robin from comment #0)
> Description of problem:
> We are using sudoHost ldap attributes with address/netmask form: xx.y.zz.0/24
> And the rules defined with this attribute are seldom functionals for our
> users.
> 
> We have to stop the sssd service, delete the sssd folder /var/lib/sssd/db,
> restart the sssd service, in order to have those rules functionals again
> with sudo.
> But the sudo rules are always lost on next reboot...
> 
> (The rules defined with sudoHost=ALL are always functionals !).
> 
> 
> Version-Release number of selected component (if applicable):
> sssd.x86_64                                          1.12.3-4.fc21
> sssd-ad.x86_64                                       1.12.3-4.fc21
> sssd-client.x86_64                                   1.12.3-4.fc21
> sssd-common.x86_64                                   1.12.3-4.fc21
> sssd-common-pac.x86_64                               1.12.3-4.fc21
> sssd-ipa.x86_64                                      1.12.3-4.fc21
> sssd-krb5.x86_64                                     1.12.3-4.fc21
> sssd-krb5-common.x86_64                              1.12.3-4.fc21
> sssd-ldap.x86_64                                     1.12.3-4.fc21
> sssd-proxy.x86_64                                    1.12.3-4.fc21
> 
> 
> How reproducible:
> Always
> 
> Steps to Reproduce:
> 1. Define a sudoRole with a sudoHost attribute as xx.y.zz.0/24 and sudoUser
> as aaa
> 2. Start a fedora machine on the network xx.y.zz.0/24
> 3. This machine sssd.conf is defined as in "Additional info"
> 4. Login as aaa on the machine
> 5. Execute a sudo command
> 
> Actual results: 
> aaa is not allowed to run sudo on xxx. This incident will be reported.
> 
> Expected results:
> The user should be able to execute the sudo command
> 
> Additional info:
> We tried to use ldap_sudo_smart_refresh_interval and
> ldap_sudo_full_refresh_interval with very small intervals without success...
> 
> The only solution we know is to stop the sssd service, delete
> /var/lib/sssd/db, and restart the service.
> We are aware of this bug for years now.
Sudo rules should be cached in sssd cache which is stored in the directory /var/lib/sssd/db. I cannot see any reason why content of this directory should be changed after restarting of computer.

If the sudo rules are cached then it should work even in offline mode or after restarting of computer.

You can find useful information how to debug sssd sudo issues in presentation [1] on the slide 18. I hope it will help you to fix your problem.
The slide recommend to enable very verbose logging in sssd. You can filter the most critical messages with simple grep command.

grep -E "\(0x00[1-9]0\)" sssd_domain.name.log

[1] http://www.freeipa.org/images/7/77/Freeipa30_SSSD_SUDO_Integration.pdf

Comment 2 Fabrice Robin 2015-02-18 13:59:08 UTC
Thanks for the info.

I may have found something in the logs.

When the sudo rule is functional, in sssd_default.log there is:
[sssd[be[default]]] [sdap_sudo_get_ip_addresses] (0x2000): Found IP address: xx.y.zz.rrr in network xx.y.zz.0/24

And when the sudo rule is not ok, there is not this line.

I delete the sssd db cache, delete the sssd logs, chkconfig off sssd, reboot the machine, start the sssd service, and then the sudo command was OK !
The sudo rule is actually functional if I start the sssd service manually after subsequent reboot. As soon as sssd service is started by systemd at boot the sudo rule is not functional anymore.

I suspect that the host ip address and network info is used by sssd to compare with sudoHost filter. And at boot (with systemd), sssd does not have these data.
(There may also have some interaction with sssd cache which explain other behavior we met).

Have you got any hint ?

Comment 3 Fabrice Robin 2015-02-18 14:21:09 UTC
I have found a workaround creating the file /etc/systemd/sssd.service.d/workaround.conf with the lines:
[Unit]
After=network.target

I suggest to add "network.target" in /lib/systemd/system/sssd.service in place of "syslog.target"

Or/And to modify sssd sources to get address/netmask info "on demand" ( Network settings (ip addresses, etc) could actually be modified live without sssd knowing !!!)

Comment 4 Lukas Slebodnik 2015-02-18 14:29:15 UTC
(In reply to Fabrice Robin from comment #3)
> I have found a workaround creating the file
> /etc/systemd/sssd.service.d/workaround.conf with the lines:
> [Unit]
> After=network.target
>
man systemd.special says:
       network.target
           This unit is supposed to indicate when network functionality is
           available, but it is only very weakly defined what that is supposed
           to mean

It would mean that sssd will not start if machine is offline.
Or did I miss something?

Comment 5 Fabrice Robin 2015-02-18 14:33:46 UTC
My bad. Forget my previous comment on "network.target". It does not work (there are indeed some weird interaction with sssd cache db. Sometimes it is ok. Sometimes not).

At this time restarting manually the sssd service is our only option.

Comment 6 Lukas Slebodnik 2015-02-18 14:41:50 UTC
(In reply to Fabrice Robin from comment #5)
> My bad. Forget my previous comment on "network.target". It does not work
> (there are indeed some weird interaction with sssd cache db. Sometimes it is
> ok. Sometimes not).
> 
> At this time restarting manually the sssd service is our only option.

Then we need to see all debug logs. sssd.conf would be useful as well.

Comment 7 Jakub Hrozek 2015-02-18 15:01:48 UTC
(In reply to Fabrice Robin from comment #5)
> My bad. Forget my previous comment on "network.target". It does not work
> (there are indeed some weird interaction with sssd cache db. Sometimes it is
> ok. Sometimes not).
> 
> At this time restarting manually the sssd service is our only option.

Can you also test if forcing SSSD to go offline and then online again with signals (see man sssd(8)) helps the situation?

Comment 8 Fabrice Robin 2015-02-18 15:06:34 UTC
Created attachment 993166 [details]
sssd logs

2 scenarii (sssd cache db and logs have been cleaned prior to the tests) with the same steps:
1- ssh apafadname@ovh203
2- sudo yum update

KO => sudo does not work (reboot)

OK => sudo works (clean cache db, logs and starting manually sssd)

The rule with "sudoHost" which should work is named "%OVH Users"

Comment 9 Fabrice Robin 2015-02-18 15:12:07 UTC
(In reply to Jakub Hrozek from comment #7)
> (In reply to Fabrice Robin from comment #5)
> > My bad. Forget my previous comment on "network.target". It does not work
> > (there are indeed some weird interaction with sssd cache db. Sometimes it is
> > ok. Sometimes not).
> > 
> > At this time restarting manually the sssd service is our only option.
> 
> Can you also test if forcing SSSD to go offline and then online again with
> signals (see man sssd(8)) helps the situation?

I tried with SIGUSR1 and next SIGUSR2 but it does not change the situation

Comment 10 Lukas Slebodnik 2015-02-18 15:42:51 UTC
(In reply to Fabrice Robin from comment #8)
> Created attachment 993166 [details]
> sssd logs
> 
> 2 scenarii (sssd cache db and logs have been cleaned prior to the tests)
> with the same steps:
> 1- ssh apafadname@ovh203
> 2- sudo yum update
> 
> KO => sudo does not work (reboot)
> 
> OK => sudo works (clean cache db, logs and starting manually sssd)
> 
> The rule with "sudoHost" which should work is named "%OVH Users"

If you followed the instruction from Comment1 you would find a reason why there is a problem with sudo rules. The grep command for the most critical messages would show you a reason.

Periodical full refresh of sudo rules failed [dp_error: 1] ([11]: Ressource temporairement non disponible)

Here is a higher context
[sssd[be[default]]] [be_resolve_server_process] (0x0080): Couldn't resolve server (ldap.domain.test), resolver returned (5)
[sssd[be[default]]] [be_resolve_server_process] (0x1000): Trying with the next one!
[sssd[be[default]]] [fo_resolve_service_send] (0x0100): Trying to resolve service 'LDAP'
[sssd[be[default]]] [get_server_status] (0x1000): Status of server 'ldap.domain.test' is 'not working'
[sssd[be[default]]] [get_port_status] (0x1000): Port status of port 0 for server '(no name)' is 'not working'
[sssd[be[default]]] [get_server_status] (0x1000): Status of server 'ldap.neticoa.ovh' is 'not working'
[sssd[be[default]]] [fo_resolve_service_send] (0x0020): No available servers for service 'LDAP'
[sssd[be[default]]] [be_resolve_server_done] (0x1000): Server resolution failed: 5
[sssd[be[default]]] [sdap_id_op_connect_done] (0x0020): Failed to connect, going offline (5 [Erreur d'entrée/sortie])
[sssd[be[default]]] [be_mark_offline] (0x2000): Going offline!
[sssd[be[default]]] [be_mark_offline] (0x2000): Initialize check_if_online_ptask.
[sssd[be[default]]] [be_ptask_create] (0x0400): Periodic task [Check if online (periodic)] was created
[sssd[be[default]]] [be_ptask_schedule] (0x0400): Task [Check if online (periodic)]: scheduling task 84 seconds from now [1424271443]
[sssd[be[default]]] [be_run_offline_cb] (0x0080): Going offline. Running callbacks.
[sssd[be[default]]] [sdap_sudo_periodical_first_refresh_done] (0x0040): Periodical full refresh of sudo rules failed [dp_error: 1] ([11]: Ressource temporairement non disponible)

Comment 11 Fabrice Robin 2015-02-18 15:48:19 UTC
During the KO scenario, when sssd goes "online" (at boot), there is a ldap request which does not search for sudoHost containing host ip address or netmask:

line 679:
[sssd[be[default]]] [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [(&(objectClass=sudoRole)(|(!(sudoHost=*))(sudoHost=ALL)(sudoHost=ovh203.neticoa.ovh)(sudoHost=ovh203)(sudoHost=+*)(|(sudoHost=*\\*)(sudoHost=*?*)(sudoHost=*\2A*)(sudoHost=*[*]*))))][ou=SUDOers,dc=neticoa,dc=lan].

whereas during the OK scenario, the same ldap request adds criteria on host ip address and netmask:
line 382:
[sssd[be[default]]] [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [(&(objectClass=sudoRole)(|(!(sudoHost=*))(sudoHost=ALL)(sudoHost=ovh203.neticoa.ovh)(sudoHost=ovh203)(sudoHost=10.2.47.203)(sudoHost=10.2.47.0/24)(sudoHost=+*)(|(sudoHost=*\\*)(sudoHost=*?*)(sudoHost=*\2A*)(sudoHost=*[*]*))))][ou=SUDOers,dc=neticoa,dc=lan].

Comment 12 Fabrice Robin 2015-02-18 15:51:54 UTC
(In reply to Lukas Slebodnik from comment #10)
> (In reply to Fabrice Robin from comment #8)
> > Created attachment 993166 [details]
> > sssd logs
> > 
> > 2 scenarii (sssd cache db and logs have been cleaned prior to the tests)
> > with the same steps:
> > 1- ssh apafadname@ovh203
> > 2- sudo yum update
> > 
> > KO => sudo does not work (reboot)
> > 
> > OK => sudo works (clean cache db, logs and starting manually sssd)
> > 
> > The rule with "sudoHost" which should work is named "%OVH Users"
> 
> If you followed the instruction from Comment1 you would find a reason why
> there is a problem with sudo rules. The grep command for the most critical
> messages would show you a reason.
> 
> Periodical full refresh of sudo rules failed [dp_error: 1] ([11]: Ressource
> temporairement non disponible)

Yes, this happens during host boot.

But when sssd go back "online" there is a bug. Look at my previous comment.

Comment 13 Fabrice Robin 2015-02-18 16:11:08 UTC
Looking at the sources, the host infos are only checked once at sssd init:
=>sdap_sudo_init
  =>sdap_sudo_get_hostinfo_send
    =>sdap_sudo_get_ip_addresses

Don't you think it should be checked each time sssd goes "online" or at each "refresh" from ldap ?

Comment 14 Lukas Slebodnik 2015-02-18 16:22:42 UTC
Thank you for analysis.

Pavel,
Could you explain it?

Comment 15 Jakub Hrozek 2015-02-18 16:44:13 UTC
(In reply to Fabrice Robin from comment #13)
> Looking at the sources, the host infos are only checked once at sssd init:
> =>sdap_sudo_init
>   =>sdap_sudo_get_hostinfo_send
>     =>sdap_sudo_get_ip_addresses
> 
> Don't you think it should be checked each time sssd goes "online" or at each
> "refresh" from ldap ?

Yes, I think it should be re-checked each time sssd goes online, because chances are it's a roaming laptop that moves between offices etc..

Comment 16 Fabrice Robin 2015-06-09 14:18:14 UTC
Do you need any help on this bug ?

Comment 17 Jakub Hrozek 2015-06-09 21:16:59 UTC
(In reply to Fabrice Robin from comment #16)
> Do you need any help on this bug ?

The best would be if Pavel Brezina could take a look at this bug.

Two questions for  you Pavel:
1) Do you agree we should do a refresh (full probably?) on transitioning from offline to online?

2) Can you see any other errors Fabrice might be facing?

Comment 18 Pavel Březina 2015-06-10 08:02:27 UTC
Hi,

ad 1) We already do full refresh when we go online.
ad 2) But we only resolve hostnames and ip addresses on boot, so if network is offline when sssd starts it fails to resolve hostnames and ip addresses and it never tries it again. We should do this when sssd goes online. So Fabrice is correct in #13.

Note: if sssd is restarted, full refresh is delayed for few seconds (I think 10) so that is probably why you don't see any change immediately after manual reboot.

Comment 19 Jakub Hrozek 2015-06-10 08:25:16 UTC
(In reply to Pavel Březina from comment #18)
Thanks for looking into this.

> ad 2) But we only resolve hostnames and ip addresses on boot, so if network
> is offline when sssd starts it fails to resolve hostnames and ip addresses
> and it never tries it again. We should do this when sssd goes online. So
> Fabrice is correct in #13.
> 

Can you file an upstream ticket with as many details as possible so we can implement it eventually?

Comment 20 Pavel Březina 2015-06-10 08:39:32 UTC
https://fedorahosted.org/sssd/ticket/2672

Comment 21 Jakub Hrozek 2015-06-10 08:48:25 UTC
Upstream ticket:
https://fedorahosted.org/sssd/ticket/2672

Comment 22 Fedora End Of Life 2015-11-04 11:30:58 UTC
This message is a reminder that Fedora 21 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 21. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '21'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 21 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 23 Fabrice Robin 2015-11-04 11:43:53 UTC
This big is still valid in Fedora 22

Comment 24 Lukas Slebodnik 2015-12-04 13:02:06 UTC
FYI. Patch is on review in upstream.

Comment 25 Lukas Slebodnik 2015-12-15 15:44:35 UTC
master:
* cb235ec146f1ba81c211f8506736edea436be28a
* 556801ec367543a8d534e55ecd11a977642bcee6
* c0000a8cc9eccdf5cd8dd72fd6e9bc09d8c7cf00
* 1ab2b07c71da6c19c3855e390d10156d598c06a2 

sssd-1-13:
* cf6f8b46cd6147fcee0534fab1c278318df58852
* bbeaec95e9814909a0bad62b31333c132558f320
* 69993374865057cf6578a69ed431dcb74d857042
* 62f97308c83378e1e82abb34894e4bddd7820fe9

Comment 26 Fedora Update System 2015-12-16 13:11:33 UTC
sssd-1.13.3-1.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2015-d47f2e33d6

Comment 27 Fedora Update System 2015-12-16 13:12:19 UTC
sssd-1.13.3-1.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2015-d4924ffeaa

Comment 28 Fedora Update System 2015-12-17 10:28:50 UTC
sssd-1.13.3-1.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'dnf --enablerepo=updates-testing update sssd'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-d4924ffeaa

Comment 29 Fedora Update System 2015-12-17 10:28:54 UTC
sssd-1.13.3-1.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'dnf --enablerepo=updates-testing update sssd'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-d47f2e33d6

Comment 30 Fedora Update System 2015-12-19 18:25:28 UTC
sssd-1.13.3-1.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.

Comment 31 Fedora Update System 2015-12-28 23:52:54 UTC
sssd-1.13.3-1.fc22 has been pushed to the Fedora 22 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.