Bug 2239137

I don't think this is the cause, but it can add some more info about what's going on: I boot another live-session and opened a tty2 to see on journal what was being logged and then it showed several coredumps.
Here is the screencast from this moment: https://geraldosimiao.fedorapeople.org/Bugzilla/F39_SOAS_beta_screencast_journalctl.webm

I tested upgrading anaconda to version anaconda-39.32.3-1.fc39, but bug still present at SOAS, can't install to HD.

Here is the error message I get after running 'liveinst' with new anaconda version:

liveuser@localhost-live:~$ liveinst
localuser:root being added to access control list
==== AUTHENTICATING FOR org.fedoraproject.pkexec.liveinst ====
Authentication is required to run the installer
Authenticating as: Live System User (liveuser)
polkit-agent-helper-1: error response to PolicyKit daemon: GDBus.Error:org.freedesktop.PolicyKit1.Error.Failed: No session for cookie
==== AUTHENTICATION FAILED ====
Error executing command as another user: Not authorized
 
This incident has been reported.
/usr/sbin/setenforce:  security_setenforce() failed:  Permission denied
vda: Failed to write 'change' to '/sys/devices/pci0000:00/0000:00:02.3/0000:04:00.0/virtio2/block/vda/uevent': Permission denied
sr0: Failed to write 'change' to '/sys/devices/pci0000:00/0000:00:1f.2/ata1/host0/target0:0:0/0:0:0:0/block/sr0/uevent': Permission denied
loop0: Failed to write 'change' to '/sys/devices/virtual/block/loop0/uevent': Permission denied
loop1: Failed to write 'change' to '/sys/devices/virtual/block/loop1/uevent': Permission denied
loop2: Failed to write 'change' to '/sys/devices/virtual/block/loop2/uevent': Permission denied
zram0: Failed to write 'change' to '/sys/devices/virtual/block/zram0/uevent': Permission denied
dm-0: Failed to write 'change' to '/sys/devices/virtual/block/dm-0/uevent': Permission denied
dm-1: Failed to write 'change' to '/sys/devices/virtual/block/dm-1/uevent': Permission denied
Traceback (most recent call last):
  File "/usr/lib64/python3.12/site-packages/gi/overrides/BlockDev.py", line 1226, in wrapped
    ret = orig_obj(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/site-packages/gi/overrides/BlockDev.py", line 866, in lvm_lvs
    return _lvm_lvs(vg_name)
           ^^^^^^^^^^^^^^^^^
gi.repository.GLib.GError: g-dbus-error-quark: GDBus.Error:org.freedesktop.DBus.Error.AccessDenied: Sender is not authorized to send message (9)
 
During handling of the above exception, another exception occurred:
 
Traceback (most recent call last):
  File "/usr/bin/anaconda-cleanup", line 82, in <module>
    devicetree.populate(cleanup_only=True)
  File "/usr/lib/python3.12/site-packages/blivet/threads.py", line 53, in run_with_lock
    return m(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/blivet/populator/populator.py", line 446, in populate
    self._populate()
  File "/usr/lib/python3.12/site-packages/blivet/threads.py", line 53, in run_with_lock
    return m(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/blivet/populator/populator.py", line 490, in _populate
    self.handle_device(dev)
  File "/usr/lib/python3.12/site-packages/blivet/threads.py", line 53, in run_with_lock
    return m(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/blivet/populator/populator.py", line 284, in handle_device
    self._add_name(name)
  File "/usr/lib/python3.12/site-packages/blivet/threads.py", line 53, in run_with_lock
    return m(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/blivet/populator/populator.py", line 146, in _add_name
    if name not in self.names:
                   ^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/blivet/threads.py", line 53, in run_with_lock
    return m(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/blivet/devicetree.py", line 148, in names
    lv_info = list(lvs_info.cache.keys())
                   ^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/blivet/static_data/lvm_info.py", line 44, in cache
    lvs = blockdev.lvm.lvs()
          ^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/site-packages/gi/overrides/BlockDev.py", line 1248, in wrapped
    raise transform[1](msg)
gi.overrides.BlockDev.LVMError: GDBus.Error:org.freedesktop.DBus.Error.AccessDenied: Sender is not authorized to send message
anaconda must be run as root.
/usr/sbin/setenforce:  security_setenforce() failed:  Permission denied

Hi,

We found the same problem in the I3Spin image. Looks like after anaconda changing from consolehelper to pkexec is the root cause. 

I managed to find one solution by adding xfce-polkit to our common Kickstart, and by adding this to our %post section at fedora-live-i3.ks:

cat > /etc/xdg/autostart/xfce-polkit.desktop <<EOF
[Desktop Entry]
Type=Application
Name=xfce-polkit
Exec=/usr/libexec/xfce-polkit
EOF

https://pagure.io/i3-sig/Fedora-i3-Spin/issue/68 was created as a tracker for members of the Sig to review the fix.

Thanks Odilon for the update. I added i3 to the Common Issue description: https://discussion.fedoraproject.org/t/90360

Thanks for pointing it out Odilon as the steps to reproduce has little to do with Sugar.

Problem still persist on new Fedora-SoaS-Live-x86_64-39-1.1.iso it seems users will not be able to install SOAS if they download the "Fedora SOAS spin"

I'm not a SoaS user, this was the patch that we applied on i3 to fix this issue. 

https://pagure.io/fedora-kickstarts/pull-request/988

Thanks! I've applied same patch https://pagure.io/fedora-kickstarts/pull-request/1006

Proposing for a freeze exception. But unless we decide to create another RC, it's too late.

+4 in https://pagure.io/fedora-qa/blocker-review/issue/1422 , marking accepted FE.

we merged this to F39, so it should be fixed in the next compose.

Unfortunately, this doesn't quite work. It seems like maybe SoaS doesn't handle XDG autostart? The changes all took effect - xfce-polkit is on the image, and the autostart file is present, but it's not run on boot. If you open a terminal and do:

/usr/libexec/xfce-polkit &
liveinst

then the installer launches OK.

As we've already started RC-1.5 and I don't know off hand how to actually get something autostarted on SoaS, we may just have to document this for final :(

Yes, I tested the last iso 1.5 and it installs only using the workaround /usr/libexec/xfce-polkit & liveinst
It seems we'll have to go for "common issues" on this.

(In reply to Adam Williamson from comment #12)
> Unfortunately, this doesn't quite work. It seems like maybe SoaS doesn't
> handle XDG autostart? The changes all took effect - xfce-polkit is on the
> image, and the autostart file is present, but it's not run on boot. If you
> open a terminal and do:
> 
> /usr/libexec/xfce-polkit &
> liveinst
> 
> then the installer launches OK.
> 
> As we've already started RC-1.5 and I don't know off hand how to actually
> get something autostarted on SoaS, we may just have to document this for
> final :(

I don't think there's particularly a different way to autostart anything on SOAS.

(In reply to Geraldo Simião from comment #13)
> Yes, I tested the last iso 1.5 and it installs only using the workaround
> /usr/libexec/xfce-polkit & liveinst
> It seems we'll have to go for "common issues" on this.

What does it mean to go for "common issues"?

It means the issue will get documented in Common Issues [1], but not fixed. The current description is at [2], I'll update it (if it's not fixed) with the current workaround. There's still an opportunity to fix this, if somebody presents a working fix and we decide to create a new RC for some reason (to be discussed today at 17:00 UTC at Go/NoGo meeting).

[1] https://discussion.fedoraproject.org/tags/c/ask/common-issues/82/none/f39
[2] https://discussion.fedoraproject.org/t/90360

(In reply to Kamil Páral from comment #16)
> It means the issue will get documented in Common Issues [1], but not fixed.
> The current description is at [2], I'll update it (if it's not fixed) with
> the current workaround. There's still an opportunity to fix this, if
> somebody presents a working fix and we decide to create a new RC for some
> reason (to be discussed today at 17:00 UTC at Go/NoGo meeting).
> 
> [1] https://discussion.fedoraproject.org/tags/c/ask/common-issues/82/none/f39
> [2] https://discussion.fedoraproject.org/t/90360

Thanks!

It's too late for solve this in F39, changing version to Rawhide (F40). The Common Issue documentation was updated.

@awilliam @kparal what would be a great way to build SOAS images without having to wait for composes, I can't find a documentation about building images with the kickstart files and I think it'll really be helpful to try and fix this issue as one can easily test changes without having to wait.

That's a good question, I'd actually also like having a good documentation on this. I can only point to [1], which is something different, but with slight tweaks, you can build a live image locally with this approach. Just make sure you include fedora-live-soas.ks (instead of qa-test-day.ks), and include a repo with your local rpms that contain the fixes that you want to test.

[1] https://fedoraproject.org/wiki/QA/Test_Days/Live_Image

Thanks for the link, I'll be sure to write up anything new I come up with.

https://fedoraproject.org/wiki/Livemedia-creator-_How_to_create_and_use_a_Live_CD is the most 'official' doc I know of. https://docs.fedoraproject.org/en-US/quick-docs/creating-and-using-a-live-installation-image/#proc_creating-and-using-live-cd is on the docs site, but is weirdly combined with the "writing an existing image" instructions (that's a completely different thing, I don't know who thought it would be a good idea to combine these) and covers the old livecd-creator tool, not the current livemedia-creator one.

I don't bother creating live images any more, I just make openQA do it for me. https://pagure.io/fedora-qa/os-autoinst-distri-fedora/blob/main/f/tests/_live_build.pm is how openQA does it.

Filed https://pagure.io/fedora-docs/quick-docs/issue/661 to try and get the docs improved...

I built an image locally with the following commands and I couldn't reproduce the issue, the installer runs, the autostart file is present and everything works as expected.

```
$ ksflatten -c fedora-live-soas.ks -o flat-fedora-live-soas.ks
$ sudo livecd-creator --verbose --config=flat-fedora-live-soas.ks --fslabel=Fedora-SOAS --cache=/var/cache/live
```

Is there something I'm missing?

official Fedora live images aren't made with livecd-creator and haven't been for quite some time. They're built with livemedia-creator , which is a bit more complicated to run (at least in a way that matches how Fedora infra builds the official images). https://fedoraproject.org/wiki/Livemedia-creator-_How_to_create_and_use_a_Live_CD is probably the best doc we have, though it's rather old and I haven't tried following it naively for years. Infra does not build in livemedia-creator's virtual machine mode, it uses the non-VM mode within a mock root.

I've built an image using the process listed in the link you shared, same results; the installer ran as expected and sugar runs fine.
The autostart files are also there.

I updated the docs.

I wonder if some update could fix the issue after F39 release. Ibiam, have you tried building the image with just the main 'fedora' repo, excluding 'updates' repo? That should yield the same result as the official F39 ISO.

Last respin (15/11/2023) still not fixed.

https://drive.google.com/drive/folders/1x70jhY5S18TB6lXPI7QYjDFuWelLm6ov

(In reply to Kamil Páral from comment #27)
> I wonder if some update could fix the issue after F39 release. Ibiam, have
> you tried building the image with just the main 'fedora' repo, excluding
> 'updates' repo? That should yield the same result as the official F39 ISO.

How do I do that?

(In reply to Geraldo Simião from comment #28)
> Last respin (15/11/2023) still not fixed.
> 
> https://drive.google.com/drive/folders/1x70jhY5S18TB6lXPI7QYjDFuWelLm6ov

I tested the image in the link and the respin works on my end. 

https://drive.google.com/file/d/14d7FbjueIky-wvtlASQo6MWhuHDl6-xK/view?usp=sharing

The terminal activity had failed to run which is why I switched views.

Can someone please confirm if this issue still persists? As I tested and it didn't.

I downloaded Fedora-SoaS-Live-x86_64-Rawhide-20231210.n.0.iso [1], and it still fails with the error from comment 2 when running liveinst.

[1] https://dl.fedoraproject.org/pub/fedora/linux/development/rawhide/Spins/x86_64/iso/

It persists on the last F39 respin too

https://drive.google.com/file/d/1wn82tpVDgpi6WLJL2UY3yHLr88aEUAWU/view?usp=drive_link F39 respin 20231215

BTW, it install only running the suggested:
sudo /usr/libexec/xfce-polkit & liveinst

(In reply to Kamil Páral from comment #32)
> I downloaded Fedora-SoaS-Live-x86_64-Rawhide-20231210.n.0.iso [1], and it
> still fails with the error from comment 2 when running liveinst.
> 
> [1]
> https://dl.fedoraproject.org/pub/fedora/linux/development/rawhide/Spins/
> x86_64/iso/

I just ran the image in a vm and F39 installs fine, I can't test on bare metal right now, are you testing in a vm? If yes, what VM?

I'm using Virtual Machine Manager.

Created attachment 2004810 [details]
saos liveinst video

That's really interesting, that it works for you.

Here's a video from a default BIOS-based VM from virt-manager, where it fails to run the installer for me.

Can you spot any differences on your side?

I now understand what the problem is, it's my fault for not taking my time to understand what the problem was and I apologize for going back and forth with everyone.

It doesn't work on my end without the work around above, I'll look at fixing it. Thanks!

The xfce-polkit startup files are present but isn't autostarted - this is limited to the file only as other autostart files aren't also run it seems -, I have a theory, but I need help understanding certain things, how is the rpm package https://src.fedoraproject.org/rpms/sugar used by the kickstart files as I noticed that polkit is specified as a package required but not xfce-polkit, I'd like to test this if I know how it's related so I can make the change necessary and point the kickstarts to my own changes and test. Thank you!

@awilliam @kparal I'd tried changing the location of the autostart file, and built an image but it never shows up in that directory, it still shows up in /etc/xdg/autostart, any idea why this is the case?

cat > $HOME/.config/autostart/xfce-polkit.desktop <<EOF
[Desktop Entry]
Type=Application
Name=xfce-polkit
Exec=/usr/libexec/xfce-polkit
EOF

Possibly because $HOME expands to something else than you expect? Or maybe the liveuser home directory is created on the fly during boot? In any case, I think you need to stick to just modifying system files, and not user files.

"Possibly because $HOME expands to something else than you expect?"

Yeah, that would be my guess. It depends where you're modifying this exactly, but if you're modifying https://pagure.io/livesys-scripts/blob/main/f/libexec/livesys/sessions.d/livesys-soas , that will get executed by root during startup, I don't recall off-hand if the liveuser home directory exists yet at that point or not. If you're modifying https://pagure.io/fedora-kickstarts/blob/main/f/fedora-soas-common.ks , that gets run in a chroot during image build, I'm pretty sure liveuser doesn't exist yet at that point. I wouldn't expect $HOME to DTRT either time.

A side note: Another workaround for this problem is to run "sudo dbus-launch liveinst", which was originally mentioned in LXDE anaconda startup issue in bug 2240162.

(In reply to Kamil Páral from comment #41)
> Possibly because $HOME expands to something else than you expect? Or maybe
> the liveuser home directory is created on the fly during boot? In any case,
> I think you need to stick to just modifying system files, and not user files.

Alright.


(In reply to Adam Williamson from comment #42)
> "Possibly because $HOME expands to something else than you expect?"
> 
> Yeah, that would be my guess. It depends where you're modifying this
> exactly, but if you're modifying
> https://pagure.io/livesys-scripts/blob/main/f/libexec/livesys/sessions.d/
> livesys-soas , that will get executed by root during startup, I don't recall
> off-hand if the liveuser home directory exists yet at that point or not. If
> you're modifying
> https://pagure.io/fedora-kickstarts/blob/main/f/fedora-soas-common.ks , that
> gets run in a chroot during image build, I'm pretty sure liveuser doesn't
> exist yet at that point. I wouldn't expect $HOME to DTRT either time.

I'm modifying https://pagure.io/fedora-kickstarts/blob/main/f/fedora-live-soas.ks, seems it also gets run in a chroot during image build.

still an issue at the last rawhide iso: Fedora-SoaS-Live-x86_64-Rawhide-20240109.n.0.iso

Proposed as a Freeze Exception for 40-beta by Fedora user geraldosimiao using the blocker tracking app because:

 If SOAS was a release-blocking desktop, this would be a blocker bug as it violates https://fedoraproject.org/wiki/Basic_Release_Criteria#Installer_must_run so I propose as a FE.

@awilliam @kparal I'd like to know if starting xfce-polkit as a systemd service would be a good fix for this, I say this because liveinst runs fine as long as xfce-polkit is running, running `/usr/libexec/xfce-polkit & liveinst` doesn't work the first time but it works the second time as it seems xfce-polkit gets started after it's run the first time so liveinst can run the second time, I say that because running it the second time opens a dialog that tells you there's an agent already running.

(In reply to Ibiam Chihurumnaya from comment #47)
> I'd like to know if starting
> xfce-polkit as a systemd service would be a good fix for this

This is how i3 spin resolved the same(?) bug:
https://pagure.io/fedora-kickstarts/pull-request/988#request_diff

So possibly you can do the same?


> running
> `/usr/libexec/xfce-polkit & liveinst` doesn't work the first time but it
> works the second time 

That's because you're running both commands at the same time (& means "start in the background"). The Common Issue has it as two separate commands, because before you start the second one, the first one is already started. Or you can do e.g. this:

/usr/libexec/xfce-polkit & sleep 2; liveinst

(In reply to Kamil Páral from comment #48)
> (In reply to Ibiam Chihurumnaya from comment #47)
> > I'd like to know if starting
> > xfce-polkit as a systemd service would be a good fix for this
> 
> This is how i3 spin resolved the same(?) bug:
> https://pagure.io/fedora-kickstarts/pull-request/988#request_diff
> 
> So possibly you can do the same?

This is what I tried first https://pagure.io/fedora-kickstarts/pull-request/1006, but for some reason it didn't autostart.

(In reply to Kamil Páral from comment #43)
> A side note: Another workaround for this problem is to run "sudo dbus-launch
> liveinst", which was originally mentioned in LXDE anaconda startup issue in
> bug 2240162.

Yeah, I tested this too and indeed it works fine to start anaconda.

(In reply to Geraldo Simião from comment #50)
> (In reply to Kamil Páral from comment #43)
> > A side note: Another workaround for this problem is to run "sudo dbus-launch
> > liveinst", which was originally mentioned in LXDE anaconda startup issue in
> > bug 2240162.
> 
> Yeah, I tested this too and indeed it works fine to start anaconda.

Yeah, I tried it too and it worked fine. Getting it to autostart is the problem.

I looked at it today and my current theory is that Sugar DE simply doesn't support xdg autostart spec. But I couldn't find any information on this subject, it's just my assumption.

So yes, the next best attempt is probably to try to create a systemd service that would start xfce-polkit, as a system (that would run even in the installed system) or user (that would be thrown away together with liveuser account) service.

This bug appears to have been reported against 'rawhide' during the Fedora Linux 40 development cycle.
Changing version to 40.

(In reply to Kamil Páral from comment #52)
> I looked at it today and my current theory is that Sugar DE simply doesn't
> support xdg autostart spec. But I couldn't find any information on this
> subject, it's just my assumption.
> 
> So yes, the next best attempt is probably to try to create a systemd service
> that would start xfce-polkit, as a system (that would run even in the
> installed system) or user (that would be thrown away together with liveuser
> account) service.

I just created an image and tried creating a systemd service which I put in /etc/systemd/system/xfce-polkit.service, and I
can confirm that the file exists but it seems the service doesn't get started.


Contents of the file;

[Unit]
Description=Simple PolicyKit authentication agent for Xfce.
Documentation=https://github.com/ncopa/xfce-polkit

Service]
ExecStart=/usr/libexec/xfce-polkit

[Install]
WantedBy=multi-user.target


systemctl shows this error; Warning: The unit file, source configuration file or drop-ins of xfce-polkit.service changed on disk.

What could be the reason why?

I'm not sure exactly how/when you went about adding that file, but there needs to be *some* kind of re-initialization done after adding it. Rebooting does it, or running `systemctl daemon-reload`.

(In reply to Adam Williamson from comment #55)
> I'm not sure exactly how/when you went about adding that file, but there
> needs to be *some* kind of re-initialization done after adding it. Rebooting
> does it, or running `systemctl daemon-reload`.

I added an entry to the fedora-live-soas kickstart file and built an image from that locally, after running `systemctl daemon-reload` the service starts but liveinst runs and never starts, it seems to freeze after showing this output `localuser: root being added to access control list`, the var/log/messages of the process doesn't really show any helpful output, these are the last few lines;

Mar 19 08:46:17 fedora systemd[1]: Starting sssd-kcm.service - SSSD Kerberos Cache Manager...
Mar 19 08:46:18 fedora systemd[1]: Started sssd-kcm.service - SSSD Kerberos Cache Manager.
Mar 19 08:46:18 fedora audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=sssd-kcm comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Mar 19 08:46:18 fedora sssd_kcm[2340]: Starting up
Mar 19 08:46:28 fedora audit[2343]: USER_ACCT pid=2343 uid=1000 auid=1000 ses=1 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:accounting grantors=pam_unix acct="liveuser" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
Mar 19 08:46:28 fedora audit[2343]: USER_CMD pid=2343 uid=1000 auid=1000 ses=1 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='cwd="/home/liveuser" cmd=736370202F70726F632F323231322F726F6F742F7661722F6C6F672F6D6573736167657320696269616D403139322E3136382E302E3139363A7E2F exe="/usr/bin/sudo" terminal=pts/0 res=success'
Mar 19 08:46:28 fedora audit[2343]: CRED_REFR pid=2343 uid=1000 auid=1000 ses=1 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:setcred grantors=pam_env,pam_unix acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
Mar 19 08:46:28 fedora audit[2343]: USER_START pid=2343 uid=1000 auid=1000 ses=1 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:session_open grantors=pam_keyinit,pam_limits,pam_keyinit,pam_limits,pam_systemd,pam_unix acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'


I'll add an ExecReload to the service file to take care of the service reboot.