Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1538066
Summary: | Fails to start after upgrade to 10.1.30: /run/mariadb missing | ||
---|---|---|---|
Product: | [Fedora] Fedora | Reporter: | Cesar Eduardo Barros <cesarb> |
Component: | mariadb | Assignee: | Michal Schorm <mschorm> |
Status: | CLOSED ERRATA | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
Severity: | high | Docs Contact: | |
Priority: | unspecified | ||
Version: | 26 | CC: | daniell1, dciabrin, dedourek, hhorak, jscalia, jstanek, linux4desktop, mbayer, mkocka, mmuzila, mschorm, praiskup, setthemfree, skpgkp1, tle |
Target Milestone: | --- | ||
Target Release: | --- | ||
Hardware: | x86_64 | ||
OS: | Linux | ||
Whiteboard: | |||
Fixed In Version: | mariadb-10.1.30-2.fc26 | Doc Type: | If docs needed, set a value |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2018-01-28 21:15:26 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
Cesar Eduardo Barros
2018-01-24 12:18:39 UTC
Manually creating /run/mariadb and changing ownership to mysql.mysql allows mariadb to start normally. However, on the next reboot, the directory again disappears. I'm seeing a similar problem in F27 after upgrading to mariadb.x86_64, version 3:10.2.10-2.fc27. In my case, /var/log/mariadb/mariadb.log shows: [ERROR] mysqld: Can't create/write to file '/run/mysqld/mariadb.pid' (Errcode: 2 "No such file or directory") And I had to manually create /run/mysqld and chown to mysql.mysql for mariadb to start up again. (In reply to Tim Elliott from comment #2) > I'm seeing a similar problem in F27 after upgrading to mariadb.x86_64, > version 3:10.2.10-2.fc27. > > In my case, /var/log/mariadb/mariadb.log shows: > > [ERROR] mysqld: Can't create/write to file '/run/mysqld/mariadb.pid' > (Errcode: 2 "No such file or directory") > > And I had to manually create /run/mysqld and chown to mysql.mysql for > mariadb to start up again. This might help you: https://bugzilla.redhat.com/show_bug.cgi?id=1483331 (In reply to John DeDourek from comment #1) > Manually creating /run/mariadb and changing ownership to mysql.mysql allows > mariadb to start normally. However, on the next reboot, the directory again > disappears. Thanks. Quick workaround that works for me: echo -e '#!/bin/bash\n\n#see https://bugzilla.redhat.com/show_bug.cgi?id=1538066\nmkdir /run/mariadb && chown mysql.mysql /run/mariadb\n' >> /usr/local/bin/mariadbfix.sh && chown root:root /usr/local/bin/mariadbfix.sh && chmod 744 /usr/local/bin/mariadbfix.sh && echo -e '[Unit]\nDescription=Fix for mariadb, see https://bugzilla.redhat.com/show_bug.cgi?id=1538066\nBefore=mariadb.service\n\n[Service]\nExecStart=/usr/local/bin/mariadbfix.sh\n\n[Install]\nWantedBy=multi-user.target\n' >> /usr/lib/systemd/system/mariadbfix.service && chown root:root /usr/lib/systemd/system/mariadbfix.service && chmod 644 /usr/lib/systemd/system/mariadbfix.service && systemctl enable mariadbfix.service It basically starts a script which creates /run/mariadb before mariadb is started. Remove the workaround with: systemctl disable mariadbfix.service && rm /usr/local/bin/mariadbfix.sh && rm /usr/lib/systemd/system/mariadbfix.service I took a deep look at what might be the cause. I found out, it is most likely caused by incorrect configuration of tmpfiles. (/usr/lib/tmpfiles.d/mariadb.conf) The upstream has recently started to ship their own, however it wasn't compatible with our downstream configuration. I fixed the isue in the SPECfile and prepared the new build. The issue *should be* solved by update to the new version. I didn't need any manual work in my test environment. Please test it and let me know, if the issues persists. mariadb-10.1.30-2.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2018-e2bc1c0aab mariadb-10.1.30-2.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-e2bc1c0aab Hi, i tried testing the fix with dnf config-manager --set-enabled updates-testing dnf update --refresh it offers to Install 1 Package Upgrade 58 Packages But mariadb- is not in the list. Do i need to take another repo for that package or just wait a bit? Dunno, but on my side, both your method and my # dnf update mariadb --enablerepo=updates-testing found the correct version. Maybe, the update did not yet reached your mirror, but I'd call it unlikely. You still can download it manually from the KOJI: https://koji.fedoraproject.org/koji/taskinfo?taskID=24423405 (yeah, a little pain but should always work) seemed to be the mirror: 3hrs later i got the packages and today i got one more mariadb-configuration. But now there is another error: [root@oc ~]# systemctl status mariadb ● mariadb.service - MariaDB 10.1 database server Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled) Active: deactivating (stop-sigterm) (Result: timeout) Process: 794 ExecStartPre=/usr/libexec/mysql-prepare-db-dir mariadb.service (code=exited, status=0/SUCCESS) Process: 718 ExecStartPre=/usr/libexec/mysql-check-socket (code=exited, status=0/SUCCESS) Main PID: 844 (mysqld) Status: "InnoDB: Error: Could not find a valid tablespace file for 'owncloud/oc_authtoken'. See http://dev.mysql.com/doc/refman/5.6/en/innodb-troubleshooting-datadict.html for how to resolve the issue." Tasks: 16 (limit: 4915) CGroup: /system.slice/mariadb.service └─844 /usr/libexec/mysqld --basedir=/usr Jan 26 09:06:33 oc.server.com systemd[1]: Starting MariaDB 10.1 database server... Jan 26 09:06:33 oc.server.com mysql-check-socket[718]: Socket file /var/lib/mysql/mysql.sock exists. Jan 26 09:06:33 oc.server.com mysql-prepare-db-dir[794]: Database MariaDB is probably initialized in /var/lib/mysql already, nothing is done. Jan 26 09:06:33 oc.server.com mysql-prepare-db-dir[794]: If this is not the case, make sure the /var/lib/mysql is empty before running mysql-prepare-db-dir. Jan 26 09:06:34 oc.server.com mysqld[844]: 2018-01-26 9:06:34 139838212134080 [Note] /usr/libexec/mysqld (mysqld 10.1.30-MariaDB) starting as process 844 ... Jan 26 09:11:34 oc.server.com systemd[1]: mariadb.service: Start operation timed out. Terminating. [root@oc ~]# ps aux | grep -i mysq mysql 844 0.1 5.1 744852 104496 ? Ssl 09:06 0:00 /usr/libexec/mysqld --basedir=/usr root 1203 0.0 0.0 119468 964 pts/0 S+ 09:14 0:00 grep --color=auto -i mysq I don't know if it is still from the broken update, or from my tries to fix it. Now it's missing a tablespace, although the file is lying in /var/lib/mysql/owncloud/oc_authtoken. This looks like restoring the backup :-( Fixed the issue. owncloud databse folder had root:root. Chowning that to mysql:mysql fixed it. Probably this was me trying to fix it. 8 drwx------. 2 root root 4096 Jan 24 17:05 owncloud chown -R mysql:mysql /var/lib/mysql/owncloud/ Now the service is starting fine also after reboot. Thanks! Glad to hear it fully works again :) mariadb-10.1.30-2.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report. I had similar issue with mariadb startup on Fedora 27. Manually creating directory /run/mariadb and setting it's ownership to mysql:mysql works but then again reboot wipe out /run directory and mariadb startup fail again. Debug messages are not very helpful. My mariadb version is mariadb-10.2.10-2.fc27.x86_64. Here is my custom mariadb startup file which survive reboot. $cat /etc/systemd/system/mariadb.service #Custom service to fix mariadb startup issue with system reboot. .include /usr/lib/systemd/system/mariadb.service [Unit] Description=MariaDB 10.2 database server After=network.target [Service] Type=Idle User=mysql Group=mysql PermissionsStartOnly=true ExecStartPre=/bin/bash -c "/bin/mkdir -p /run/mariadb && /bin/chown mysql:mysql /run/mariadb" $systemctl daemon-reload (In reply to Daniel Laczi from comment #3) > (In reply to Tim Elliott from comment #2) > > I'm seeing a similar problem in F27 after upgrading to mariadb.x86_64, > > version 3:10.2.10-2.fc27. > > > > In my case, /var/log/mariadb/mariadb.log shows: > > > > [ERROR] mysqld: Can't create/write to file '/run/mysqld/mariadb.pid' > > (Errcode: 2 "No such file or directory") > > > > And I had to manually create /run/mysqld and chown to mysql.mysql for > > mariadb to start up again. > > This might help you: https://bugzilla.redhat.com/show_bug.cgi?id=1483331 Just wanted to give you a belated thanks. I do have a change in /etc/mariadb-server.cnf (to change the datadir setting out of the root filesystem) and I had to merge my config with the .rpmnew version. Everything works now. (In reply to Sunil P from comment #14) > I had similar issue with mariadb startup on Fedora 27. Manually creating > directory /run/mariadb and setting it's ownership to mysql:mysql works but > then again reboot wipe out /run directory and mariadb startup fail again. > Debug messages are not very helpful. > > My mariadb version is mariadb-10.2.10-2.fc27.x86_64. > > Here is my custom mariadb startup file which survive reboot. > > $cat /etc/systemd/system/mariadb.service > #Custom service to fix mariadb startup issue with system reboot. > .include /usr/lib/systemd/system/mariadb.service > [Unit] > Description=MariaDB 10.2 database server > After=network.target > > [Service] > Type=Idle > User=mysql > Group=mysql > PermissionsStartOnly=true > ExecStartPre=/bin/bash -c "/bin/mkdir -p /run/mariadb && /bin/chown > mysql:mysql /run/mariadb" > > $systemctl daemon-reload It was working but fedora update broke this setup as it doesn't understand Type parameter and doesn't start mariadb. /etc/systemd/system/mariadb.service:8: Failed to parse service type, ignoring: Idle Here is corrected one which work with latest Fedora and survive reboot as of now. $cat /etc/systemd/system/mariadb.service #Custom service to fix mariadb startup issue with system reboot. .include /usr/lib/systemd/system/mariadb.service [Unit] Description=MariaDB 10.2 database server After=network.target [Service] User=mysql Group=mysql PermissionsStartOnly=true ExecStartPre=/bin/bash -c "/bin/mkdir -p /run/mariadb && /bin/chown mysql:mysql /run/mariadb" $systemctl daemon-reload |