Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1756059 - Podman errors with: cannot rm state directory `.....` not empty
Summary: Podman errors with: cannot rm state directory `.....` not empty
Keywords:
Status: CLOSED DUPLICATE of bug 1753328
Alias: None
Product: Fedora
Classification: Fedora
Component: podman
Version: 31
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
Assignee: Matthew Heon
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-09-26 16:45 UTC by Ankur Sinha (FranciscoD)
Modified: 2019-10-01 12:35 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-10-01 12:35:20 UTC
Type: Bug
Embargoed:

Attachments (Terms of Use)

Description Ankur Sinha (FranciscoD) 2019-09-26 16:45:51 UTC 
Description of problem:
I'm working on the Fedora documentation, which uses podman to run antora. An example build script is here:

https://pagure.io/fedora-docs/quick-docs/blob/master/f/build.sh

In essence, it calls:	
podman run --rm -it -v $(pwd):/antora:z "docker.io/antora/antora" --html-url-extension-style=indexify site.yml


A lot of the times, but not always, this errors:
$ podman run --rm -it -v $(pwd):/antora:z "docker.io/antora/antora" --html-url-extension-style=indexify site.yml

2019-09-26T16:41:42.000845027Z: cannot rm state directory '/run/user/1000/crun/cd0681394b239fed85eb2fe3c30dd0bd0dc43c87058b19ad85e10e6eb0256052': Directory not empty
ERRO[0002] Error removing container cd0681394b239fed85eb2fe3c30dd0bd0dc43c87058b19ad85e10e6eb0256052: error cleaning up container cd0681394b239fed85eb2fe3c30dd0bd0dc43c87058b19ad85e10e6eb0256052: error removing container cd0681394b239fed85eb2fe3c30dd0bd0dc43c87058b19ad85e10e6eb0256052 from runtime: `/usr/bin/crun delete --force cd0681394b239fed85eb2fe3c30dd0bd0dc43c87058b19ad85e10e6eb0256052` failed: exit status 1 


On inspection, we see that a socket file still exists in the directory:

$ tree /run/user/1000/crun/cd0681394b239fed85eb2fe3c30dd0bd0dc43c87058b19ad85e10e6eb0256052

/run/user/1000/crun/cd0681394b239fed85eb2fe3c30dd0bd0dc43c87058b19ad85e10e6eb0256052
└── notify
    └── notify


$ file /run/user/1000/crun/cd0681394b239fed85eb2fe3c30dd0bd0dc43c87058b19ad85e10e6eb0256052/notify/notify 
/run/user/1000/crun/cd0681394b239fed85eb2fe3c30dd0bd0dc43c87058b19ad85e10e6eb0256052/notify/notify: socket


Version-Release number of selected component (if applicable):
$ rpm -q podman crun
podman-1.5.1-3.git0005792.fc31.x86_64
crun-0.9.1-1.fc31.x86_64


How reproducible:
Almost always, not not always (I've had a run or two where it didn't error)


Steps to Reproduce:
1. git clone https://pagure.io/Fedora-Council/council-docs.git
2. cd council-docs
3. ./build.sh (a number of times maybe)

Actual results:
Errors as given above.

Expected results:
Should not error

Additional info:
Not sure, other docs users on F30 say they've not seen the issue yet.
Comment 1 Daniel Walsh 2019-09-28 06:14:07 UTC 
Matt this looks like some of the sd_notify stuff that was just fixed in podman?
Comment 2 Parag Nemade 2019-09-30 05:36:49 UTC 
Ankur,

I think if you update podman package from koji https://koji.fedoraproject.org/koji/buildinfo?buildID=1390091 then podman will work for you on Fedora 31.
If its still not working then try to rm ~/.config/containers/libpod.conf and run your ./build.sh
Comment 3 Ankur Sinha (FranciscoD) 2019-09-30 22:18:49 UTC 
Hello,

No luck there either from the looks of it:

$ rm ~/.config/containers/libpod.conf -f

$ rpm -q podman
podman-1.6.0-0.2.rc2.git9181c65.fc31.x86_64

$ ./build.sh 

This build script is using Podman to run the build in an isolated environment.

WARN[0000] Error initializing configured OCI runtime runc: no valid executable found for OCI runtime runc: invalid argument 
2019-09-30T22:16:23.000681096Z: cannot rm state directory '/run/user/1000/crun/68ac7a64119049762e74a6ea25606f9233c0afa320179bc5dc83bb50490c7fa4': Directory not empty
ERRO[0002] Error removing container 68ac7a64119049762e74a6ea25606f9233c0afa320179bc5dc83bb50490c7fa4: error cleaning up container 68ac7a64119049762e74a6ea25606f9233c0afa320179bc5dc83bb50490c7fa4: error removing container 68ac7a64119049762e74a6ea25606f9233c0afa320179bc5dc83bb50490c7fa4 from runtime: `/usr/bin/crun delete --force 68ac7a64119049762e74a6ea25606f9233c0afa320179bc5dc83bb50490c7fa4` failed: exit status 1 



Is there some other configuration or package I should be modifying/updating too?
Comment 4 Ankur Sinha (FranciscoD) 2019-09-30 22:20:04 UTC 
$ tree /run/user/1000/crun/68ac7a64119049762e74a6ea25606f9233c0afa320179bc5dc83bb50490c7fa4
/run/user/1000/crun/68ac7a64119049762e74a6ea25606f9233c0afa320179bc5dc83bb50490c7fa4
└── notify
    └── notify

1 directory, 1 file

$ file /run/user/1000/crun/68ac7a64119049762e74a6ea25606f9233c0afa320179bc5dc83bb50490c7fa4/notify/notify 
/run/user/1000/crun/68ac7a64119049762e74a6ea25606f9233c0afa320179bc5dc83bb50490c7fa4/notify/notify: socket
Comment 5 Matthew Heon 2019-09-30 22:22:21 UTC 
@Dan - I feel like crun just added support for sdnotify?

Also, Podman fixing the bug that unconditionally turned it off probably has not helped.
Comment 6 Parag Nemade 2019-10-01 04:42:53 UTC 
Strange I cannot reproduce your problem again on any of Fedora 31 systems provided I have podman installed from koji.
Comment 7 Ankur Sinha (FranciscoD) 2019-10-01 09:48:37 UTC 
Do I need to restart the system or any services when podman is updated, maybe?
Comment 8 Giuseppe Scrivano 2019-10-01 11:30:58 UTC 
can it be the gnome issue where NOTIFY_SOCKET is always defined?

I think it can be a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=1753328
Comment 9 Ankur Sinha (FranciscoD) 2019-10-01 12:31:22 UTC 
(In reply to Giuseppe Scrivano from comment #8)
> can it be the gnome issue where NOTIFY_SOCKET is always defined?
> 
> I think it can be a duplicate of
> https://bugzilla.redhat.com/show_bug.cgi?id=1753328

That does seem to be the case. I rebooted, ensured that NOTIFY_SOCKET was not set and now podman seems to run just fine.

I still get this warning, but it doesn't break anything:
WARN[0000] Error initializing configured OCI runtime runc: no valid executable found for OCI runtime runc: invalid argument 

Thanks for the help, everyone. Please close this as a duplicate if that's the correct resolution here.
Comment 10 Giuseppe Scrivano 2019-10-01 12:35:20 UTC 

*** This bug has been marked as a duplicate of bug 1753328 ***
Note You need to log in before you can comment on or make changes to this bug.