Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1419040 - The change to /var/log mounting breaks the running of services that require a folder in /var/log created at docker build time
Summary: The change to /var/log mounting breaks the running of services that require a...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: oci-systemd-hook
Version: 7.3
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Lokesh Mandvekar
QA Contact: Martin Jenner
URL:
Whiteboard:
Depends On:
Blocks: 1423502
TreeView+ depends on / blocked
 
Reported: 2017-02-03 14:04 UTC by James Hogarth
Modified: 2017-04-12 14:51 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: oci-systemd-hook was running as container_t, (svirt_lxc_net_t) and should not have been. Consequence: This caused systemctl is-active to show failures when mariadb tried to start within a container and other selinux errors. Fix: This was corrected in oci-systemd-hook-0.1.6-1.gitfe22236.el7.x86_64. Result: mariadb containers now start as expected.
Clone Of:
Environment:
Last Closed: 2017-04-12 14:51:02 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github projectatomic oci-systemd-hook issues 46 0 None None None 2017-02-03 14:40:46 UTC
Red Hat Bugzilla 1412728 0 high CLOSED Bind mount on /var/log seems to be over shadowed 2023-09-12 01:13:07 UTC
Red Hat Product Errata RHBA-2017:0949 0 normal SHIPPED_LIVE oci-systemd-hook bug fix update 2017-04-12 18:32:45 UTC

Internal Links: 1412728 1423502

Description James Hogarth 2017-02-03 14:04:00 UTC 
Description of problem:
In the previous version a build script that created a mariadb docker container worked fine. The dockerfile installs mariadb.

The present version results in the container having a minimal /var/log and the /var/log/mariadb that was created by the mariadb-server rpm doesn't exist ar runtime. As a result the mariadb service fails to respond.

Version-Release number of selected component (if applicable):
oci-systemd-hook-0.1.4-9.git671c428.el7.x86_64 <-- broken
oci-systemd-hook-0.1.4-6.git337078c.el7.c86_64 <-- works

How reproducible:
always

Steps to Reproduce:
cat > Dockerfile.mariadb << EOF
FROM centos:latest
STOPSIGNAL SIGRTMIN+3
 
RUN yum -y install mariadb-server && yum clean all
 
RUN systemctl enable mariadb
 
VOLUME /var/lib/mysql
 
CMD ["/sbin/init"]
EOF
 
docker volume create --name localtest-mdb
docker build -f Dockerfile.mariadb -t localtest-mdb .
docker run -dt -v localtest-mdb:/var/lib/mysql --name localtest-mdb localtest-mdb
docker exec -t localtest-mdb /bin/bash -c 'for i in {1..30}; do if systemctl is-active mariadb ; then break  ; else sleep 1 ; fi done;'
docker exec -t localtest-mdb mysql -e "GRANT ALL PRIVILEGES ON *.* TO 'testuser'@'%' IDENTIFIED BY 'testpassword' WITH GRANT OPTION;"
docker stop localtest-mdb
docker rm localtest-mdb

Actual results:
systemctl is-active shows failed in the container when mariadb tries to start rather than active
checking the logs it cannot find /var/log/mariadb , which was created in the docker build as a result of installing the RPM

Expected results:
On a fedora system (or older RHEL7 system, but the docker 1.12 in RHEL extras has -8 as a minimal dependency so I can't cleanly revert to the older hook) systemctl is-active shows mariadb becoming active fine.


Additional info:
Note that using the oci-systemd-hook in fedora from here shows the regression:
https://koji.fedoraproject.org/koji/buildinfo?buildID=837992

On my fedora test system (and older rhel7 installs) where I have the older hook:
[root@7bfbfd09e947 /]# ls -l /var/log/
total 32
drwxr-xr-x. 1 root  root    186 Dec 14 14:59 anaconda
-rw-------. 1 root  utmp      0 Dec 14 14:58 btmp
drwxr-xr-x. 1 root  root     64 Feb  3 14:01 journal
-rw-r--r--. 1 root  root  23944 Feb  3 13:46 lastlog
drwxr-x---. 1 mysql mysql    22 Feb  3 13:46 mariadb
-rw-------. 1 root  root      0 Dec 14 14:58 tallylog
-rw-rw-r--. 1 root  utmp      0 Dec 14 14:58 wtmp
-rw-------. 1 root  root   4446 Feb  3 13:46 yum.log

On my fedora test system with the newer hook from koji (and this matches the current behaviour on rhel7):
[root@43f86ed6dacd /]# ls -l /var/log
total 4
-rw-------. 1 root utmp              0 Feb  3 14:02 btmp
drwxr-sr-x. 3 root systemd-journal  60 Feb  3 14:02 journal
-rw-rw-r--. 1 root utmp            384 Feb  3 14:02 wtmp

It looks like the original data isn't being copied in
Comment 1 James Hogarth 2017-02-03 14:14:37 UTC 
Okay I just tested using rpm to force installing the older package and accepting the dependency issues for now ...

I can definitely confirm that the build&run works then and that this update is a regression that breaks anything that expects something in /var/log already to exist - such as mariadb
Comment 3 James Hogarth 2017-02-09 12:00:00 UTC 
Confirming in my tests that the recent build in Fedora fixes this:

https://koji.fedoraproject.org/koji/buildinfo?buildID=839423

If this could be built and rolled out to RHEL Extras it'd be appreciated.
Comment 4 Robert Scheck 2017-02-17 13:24:44 UTC 
Aside of this, the update of oci-systemd-hook to 0.1.4-9.git671c428.el7.x86_64
leads to bug #1423502 here, while 0.1.5-1.git16f7c8a.fc25.x86_64 (as mentioned
in comment #3) indeed fixes it, thus I also would like to see the fix in RHEL 
Extras.

Cross-filed case 01794554 on the Red Hat customer portal for both issues.
Comment 5 Robert Scheck 2017-02-20 14:04:12 UTC 
Btw, let me clarify this: The SELinux errors show up with both packages,

 - oci-systemd-hook-0.1.4-9.git671c428.el7.x86_64
 - oci-systemd-hook-0.1.5-1.git16f7c8a.fc25.x86_64

but not with the old oci-systemd-hook-0.1.4-6.git337078c.el7.c86_64. However,
yes, oci-systemd-hook-0.1.5-1.git16f7c8a.fc25.x86_64, makes MariaDB instances
working again.
Comment 6 Robert Scheck 2017-02-20 21:12:29 UTC 
As conclusion on bug #1423502 comment #5, 0.1.5-1.git16f7c8a.fc25.x86_64
for RHEL Extras is not the ultimate solution, oci-systemd-hook still needs
to be fixed instead (to avoid the behavior described in bug #1423502).
Comment 7 Daniel Walsh 2017-03-12 11:48:56 UTC 
Fixed in oci-systemd-hook-0.1.6-1.gitfe22236.el7
Comment 8 Tom Sweeney 2017-03-25 20:27:56 UTC 
(In reply to Daniel Walsh from comment #7)
> Fixed in oci-systemd-hook-0.1.6-1.gitfe22236.el7

It appears to be fixed.  Test results follow.  Dan I'll touch base with you Monday about closing.

[root@rhel73bz ~]# cat > Dockerfile.mariadb << EOF
> FROM centos:latest
> STOPSIGNAL SIGRTMIN+3
>  
> RUN yum -y install mariadb-server && yum clean all
>  
> RUN systemctl enable mariadb
>  
> VOLUME /var/lib/mysql
>  
> CMD ["/sbin/init"]
> EOF
[root@rhel73bz ~]# docker volume create --name localtest-mdb
localtest-mdb
[root@rhel73bz ~]# docker build -f Dockerfile.mariadb -t localtest-mdb .
Sending build context to Docker daemon 14.85 kB
Step 1 : FROM centos:latest
 ---> 98d35105a391
Step 2 : STOPSIGNAL SIGRTMIN+3
 ---> Using cache
 ---> d21037da37ed
Step 3 : RUN yum -y install mariadb-server && yum clean all
 ---> Using cache
 ---> 4440f237aa2b
Step 4 : RUN systemctl enable mariadb
 ---> Using cache
 ---> 007cbeb7dcff
Step 5 : VOLUME /var/lib/mysql
 ---> Using cache
 ---> dc7109037dbf
Step 6 : CMD /sbin/init
 ---> Using cache
 ---> 53e981575dd6
Successfully built 53e981575dd6
[root@rhel73bz ~]# docker run -dt -v localtest-mdb:/var/lib/mysql --name localtest-mdb localtest-mdb
e54bcedff15d675fecba00e74a302a2a0948dc99c17afd575992ee501675a114
[root@rhel73bz ~]# docker exec -t localtest-mdb /bin/bash -c 'for i in {1..30}; do if systemctl is-active mariadb ; then break  ; else sleep 1 ; fi done;'
activating
activating
activating
active
[root@rhel73bz ~]# docker exec -t localtest-mdb mysql -e "GRANT ALL PRIVILEGES ON *.* TO 'testuser'@'%' IDENTIFIED BY 'testpassword' WITH GRANT OPTION;"
[root@rhel73bz ~]# docker stop localtest-mdb
localtest-mdb
[root@rhel73bz ~]# docker rm localtest-mdb
localtest-mdb
[root@rhel73bz ~]# uname -a
Linux rhel73bz.localdomain 3.10.0-625.el7.x86_64 #1 SMP Thu Mar 23 11:04:30 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux
[root@rhel73bz ~]# 
[root@rhel73bz ~]# 
[root@rhel73bz ~]# rpm -qa | grep oci-systemd-hook
oci-systemd-hook-0.1.6-1.gitfe22236.el7.x86_64
Comment 10 David Darrah/Red Hat QE 2017-03-28 21:03:56 UTC 
Verified as above on RHEL with remote db service access across the network.
Comment 13 errata-xmlrpc 2017-04-12 14:51:02 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:0949
Note You need to log in before you can comment on or make changes to this bug.