Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 834746 - F17 update causes separate /usr partition to be mounted read-only
Summary: F17 update causes separate /usr partition to be mounted read-only
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Fedora
Classification: Fedora
Component: anaconda
Version: 17
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Anaconda Maintenance Team
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-06-23 06:15 UTC by bob mckay
Modified: 2013-05-07 23:39 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-05-07 23:39:56 UTC
Type: Bug
Embargoed:

Attachments (Terms of Use)
Original anaconda ks file (deleted)
2012-07-04 12:29 UTC, bob mckay 		no flags Details
F15 install log (deleted)
2012-07-04 12:29 UTC, bob mckay 		no flags Details
F15 install syslog (deleted)
2012-07-04 12:30 UTC, bob mckay 		no flags Details
F16->F17 upgrade log (deleted)
2012-07-04 12:30 UTC, bob mckay 		no flags Details
F16->F17 upgrade syslog (deleted)
2012-07-04 12:31 UTC, bob mckay 		no flags Details
Anaconda F16->F17 ifcfg log (deleted)
2012-07-05 00:32 UTC, bob mckay 		no flags Details
Anaconda F16->F17 log (deleted)
2012-07-05 00:33 UTC, bob mckay 		no flags Details
Anaconda F16->F17 program log (deleted)
2012-07-05 00:33 UTC, bob mckay 		no flags Details
Anaconda F16->F17 storage log (deleted)
2012-07-05 00:37 UTC, bob mckay 		no flags Details
Anaconda F16->F17 syslog (deleted)
2012-07-05 00:38 UTC, bob mckay 		no flags Details
Anaconda F16->F17 xlog (deleted)
2012-07-05 00:38 UTC, bob mckay 		no flags Details
Anaconda F16->F17 yum log (deleted)
2012-07-05 00:39 UTC, bob mckay 		no flags Details
View All

Description bob mckay 2012-06-23 06:15:12 UTC 
Description of problem:
When upgrading an F16 system with a separate /usr partition to F17, the /usr partition gets mounted readonly on first reboot, which breaks all sorts of things... Sorry, I don't really know whether this is a bug in anaconda or something else. I guess it may well be related to bug 804913 but I'm not sure.

Version-Release number of selected component (if applicable):
F17 upgrade tools (DVD and preupgrade routes for certain). 

How reproducible:
Reliably (about 6 systems so far, both DVD and preupgrade upgrades).

Steps to Reproduce:
1. Install F16
2. Upgrade to F17 (DVD or preupgrade)
3. Reboot
  
Actual results:
/usr is mounted ro

Expected results:
/usr is mounted rw

Additional info:
For anyone hitting this problem, the following sequence seems to fix the problem
mount -o rw/remount /usr
yum --skip-broken distro-sync
(if your yum configuration is really broken you may need to remove some packages to get this to work - I have had to remove some sane and cups library packages, the ones that actually generate the failure messages, to get through this)
reboot
After the reboot, /usr seems to mount rw as expected.

Symptoms:

###############################################################################

%yum update kernel:

Transaction Check Error:
  installing package kernel-3.4.3-1.fc17.x86_64 needs 98MB on the /usr filesystem

Error Summary
-------------
Disk Requirements:
  At least 98MB more space needed on the /usr filesystem.


###############################################################################
df output:
/dev/mapper/vg_sc2-lv_usr       20415616   8343620  11047996  44% /usr
(i.e. /usr has over 8GB space available).
Comment 1 Mads Kiilerich 2012-06-23 11:23:20 UTC 
Please attach all the anaconda log files.
Comment 2 bob mckay 2012-07-04 12:29:16 UTC 
Created attachment 596205 [details]
Original anaconda ks file

Original anaconda kickstart file. From the date, this was probably originally an F15 install, which would have been upgraded, probably by preupgrade, to F16, and then upgraded again to F17, this time via DVD because I had previously been bitten by preupgrade bugs with F17. Unfortunately the F15->F16 logs appear to have been overwritten by the latest upgrade.
Comment 3 bob mckay 2012-07-04 12:29:48 UTC 
Created attachment 596206 [details]
F15 install log
Comment 4 bob mckay 2012-07-04 12:30:18 UTC 
Created attachment 596207 [details]
F15 install syslog
Comment 5 bob mckay 2012-07-04 12:30:52 UTC 
Created attachment 596208 [details]
F16->F17 upgrade log
Comment 6 bob mckay 2012-07-04 12:31:30 UTC 
Created attachment 596209 [details]
F16->F17 upgrade syslog
Comment 7 Mads Kiilerich 2012-07-04 21:38:26 UTC 
You don't have any anaconda logs - https://fedoraproject.org/wiki/Anaconda/Logging#Anaconda_logs_on_the_running_system ?

dmesg from the first ro boot would perhaps also tell something.

What changes do the yum distrosync do - and which of them will make it work?

When and why do you get the yum kernel update error? Is it right? If not then it seems to be something that should be filed as a separate bug.
Comment 8 bob mckay 2012-07-05 00:32:00 UTC 
Created attachment 596299 [details]
Anaconda F16->F17 ifcfg log
Comment 9 bob mckay 2012-07-05 00:33:05 UTC 
Created attachment 596300 [details]
Anaconda F16->F17 log
Comment 10 bob mckay 2012-07-05 00:33:39 UTC 
Created attachment 596301 [details]
Anaconda F16->F17 program log
Comment 11 bob mckay 2012-07-05 00:37:21 UTC 
Created attachment 596302 [details]
Anaconda F16->F17 storage log
Comment 12 bob mckay 2012-07-05 00:38:01 UTC 
Created attachment 596303 [details]
Anaconda F16->F17 syslog
Comment 13 bob mckay 2012-07-05 00:38:35 UTC 
Created attachment 596304 [details]
Anaconda F16->F17 xlog
Comment 14 bob mckay 2012-07-05 00:39:04 UTC 
Created attachment 596305 [details]
Anaconda F16->F17 yum log
Comment 15 bob mckay 2012-07-05 01:24:26 UTC 
(In reply to comment #7)
> You don't have any anaconda logs -
> https://fedoraproject.org/wiki/Anaconda/
> Logging#Anaconda_logs_on_the_running_system ?

Sorry Mads, my misunderstanding, I have supplied these now. 

> 
> dmesg from the first ro boot would perhaps also tell something.

I'm sorry, I don't think I'm going to be able to supply this. I didn't think to do it before I fixed the systems which failed. I only have three more systems to upgrade, and unfortunately they are mission critical (which is why I have left them to last). I will have to upgrade them by preupgrade (which seems to be more reliable) rather than a DVD upgrade. 

> 
> When and why do you get the yum kernel update error? Is it right? If not
> then it seems to be something that should be filed as a separate bug.

This specific error is what I was seeing when I first rebooted the system. Of course, the first thing I tried to do was a full yum update - which fails (with the misleading message about /usr space) because /usr is ro. If I'd done something else first, I may have seen a different consequence, but I thought it was worth showing because I think most people are likely to do a yum update soon after upgrading the system, so it's likely to be the first time they encounter problems resulting from the ro mount. I don't think it's a separate bug, it is a direct consequence of the ro mount of /usr, and goes away as soon as that is fixed.

> 
> What changes do the yum distrosync do - and which of them will make it work?

Even once /usr is mounted rw, yum update throws hundreds of errors about duplicate packages (seems a lot of f16 packages are not removed) and then aborts with errors about sane backends and cups libraries. Manually removing the sane and cups culprits gets the system to the stage where yum --skip-broken distro-sync works (yum --skip-broken update still fails, I think it is probably just bug 820351, but not sure). The distrosync does hundreds of updates; I'm sorry, I have no way to find out which one actually fixes the ro mount problem. Finally, I manually restore the removed sane backends and cups libraries. My memory about this is a little hazy because once I encountered so many scary problems with DVD upgrade, I decided to go back to using preupgrade - it also has problems, but they seem to be less serious.
Comment 16 bob mckay 2012-07-05 01:39:02 UTC 
(In reply to comment #15)
> 
> Even once /usr is mounted rw, yum update throws hundreds of errors about
> duplicate packages (seems a lot of f16 packages are not removed) and then
> aborts with errors about sane backends and cups libraries. 

Checking further, this looks to be bug 826621. That is, the ro mount of /usr is a new bug, but once /usr is remounted rw, bug 826621 manifests itself.
Comment 17 Andrew Wasielewski 2012-07-10 22:17:28 UTC 
I seem to have the same problem.  I have /usr on separate LVM partition.  /usr-move has completed successfully but DVD x86_64 FC16->17 upgrade failed, I think due to lack of space on /usr.  Upgrade DVD now doesn't recognise existing FC16 installation (see https://bugzilla.redhat.com/show_bug.cgi?id=835307).

A complicating factor is that /usr now mounts read-only, as far as I can see as a consequence of /usr-move.  Tried editing fstab without effect.  mount -o rw/remount /usr doesn't work either - I get 

mount: /dev/mapper/VolGroup00-LogVol02 already mounted or /usr busy
mount: according to mtab, /dev/mapper/VolGroup00-LogVol02 is already mounted on /usr

and still mounts ro at next reboot.
Comment 18 Mads Kiilerich 2012-07-31 20:38:48 UTC 
It seems like updating from f16 with a separete /usr simply isn't very well supported. (Preupgrade might be convenient, but it is also a moving target and I consider it much more fragile than upgrading from a media.)

My first step to diagnose the ro mount would be to try to boot from a live / rescue disk and verify that the /usr partition really can be mounted rw - and perhaps also run a fsck.

The reason to why it is mounted ro is somewhere on your disk.
Comment 19 Brian Lane 2013-05-07 23:39:56 UTC 
F17 is long done and Anaconda no longer handles upgrades.
Note You need to log in before you can comment on or make changes to this bug.