Friday, June 26, 2009

Fedora 10 -> Fedora 11 upgrade problems

After the last debacle [1, 2] associated with trying a yum upgrade from Fedora 9 to Fedora 10 on my x86_64 Opteron workstation I was sure that this time would be different. I dutifully downloaded and burned a x86_64 installation disk for my Opteron and a i686 for my old home PC.

Neither upgrade worked properly.

I had not run yum update on my home Fedora 10 machine since Fedora 11 was released on June 9 2009. I attempted the DVD upgrade on June 19, left it happily (if slowly) updating the ~1200 packages it wanted to update, only the return to a fatal error message regarding an unhandled exception. On reboot a single Fedora 11 kernel was available which started booting but failed when trying to go to runlevel 5. Reading the Common F11 bugs page indicated that my nvidia AGP card might be the problem, so modifying the kernel boot options as suggested allowed me to boot successfully. A large "yum update" (1.2 GB) was then necessary, but I finally had a working system.

For the Opteron I made sure my Fedora 10 installation was fully updated on the morning of 23rd June, and just before leaving working in the evening I stuck the DVD in the drive, hit the upgrade option when it appeared and left it running overnight. When I returned the upgrade claimed to have completed itself successfully and told me to reboot.

On rebooting grub went straight to a empty command line, without the usual boot menu. Clearly the grub-install that is run every time you update the kernel had not run successfully. As I know /boot is the first partition on my first hard drive I can still find the kernel and boot by doing the following on the grub command line:

root (hd0,0)
kernel /vmlin[hit tab] [select appropriate file] [return]
initrd /initrd[hit tab] [select appropriate file] [return]
boot

I did not use the standard Fedora kernel options "rhgb quiet" as I wanted to watch the services start up to see if there were any problems. Indeed, once the kernel started booting I saw that a variety of services where failing to start, complaining of shared library problems. A classic sign of a incomplete upgrade. Great!

In fact the boot process stalled completely, so I resorted to using the Fedora 11 disk in rescue mode. chrooting into the system once it had booted.

I checked that /boot/grub/grub.conf looked sensible, ran "grub-install /dev/sda" fix the grub problem, and then attempted "yum list updates", only to find that yum wouldn't run as python complained that it could not find the yum module.

"rpm -qa | grep yum" revealed that yum was still a Fedora 10 version, version 3.23. Mounting the Fedora 11 disk (mount -t iso9660 /dev/scd0 /media) revealed that the yum version on the Fedora 11 install disk is 3.22, so clearly the upgrade had failed to update yum because the recently updated Fedora 10 version had a higher version number than the Fedora 11 disk created only a few weeks before! In fact a large number of packages had not been updated to Fedora 11. Luckily rpm had been updated (if rpm was not working I'd have been forced to do a full system restore). Running

rpm -Uvh /media/Packages/yum* --oldpackage

successfully updated yum to the Fedora 11 version, at which point I could run "yum update" and start the process of getting a truly upgrading system underway.

With hindsight my mistake was updating my Fedora 10 system after Fedora 11 had been released. When doing yum upgrades it is a good idea to have your system as closely matching the new version of Fedora as possible, but this appears not to be the case for a DVD-based upgrade.

I'm still surprised that the DVD-based Fedora upgrade doesn't think to correct for version mismatches - you'd think it'd be logical for any f11 package to automatically obsolete a f10 package, even if the version numbers of the f10 package are higher. Or the upgrade could simple run rpm with the "--oldpackage" command line option.

Overall I have little satisfaction with the upgrading process under recent versions of Fedora. Web-based yum upgrades used to work very well, at least between versions ~6 and 8, but with Fedora 9 -> 10 and now with Fedora 10 -> 11 neither web-based or disk based upgrades has worked without significant problems.

Admittedly I've also experienced occasional severe problems with upgrades between different versions of Ubuntu when I was experimenting with it, so this is not a problem unique to Fedora.

The alternative - moving to a long-lifespan distribution such as RHEL or a free clone of it like Scientific Linux - simply replaces upgrade problems with equally severe out-of-date software problems (a friend who relies of the department's sysadmins to maintain his Scientific Linux distribution has had terrible problems with being stuck with old buggy versions of important software that require major version changes to fix, which the standard updates in RHEL/SL don't often do).

The only pro-active solution I can see is to become more involved with testing process on Fedora, and to have a dedicated test machine that I can easily revert to its original state when upgrade tests fail badly.

No comments: