Last Saturday, I decided it was time to switch my NAS server from 22.04 LTS to 24.04 LTS. I\u2019ve been putting it off for ages, worried that the upgrade might not go as planned and something could go wrong. Since 24.04 is already in its fourth point release, I figured the risks should be manageable and it\u2019s time to take the plunge.<\/p>\n\n\n\n
I backup my system nightly so the insurance was in place. After performing a final regular update to the system, I started with the following:<\/p>\n\n\n\n
sudo apt update && sudo apt upgrade && sudo apt dist-upgrade<\/code><\/pre>\n\n\n\nI then rebooted the system and executed:<\/p>\n\n\n\n
sudo do-release-upgrade<\/code><\/pre>\n\n\n\nAfter answering a few questions to save my custom configuration files for different services, it said the upgrade was done. I then rebooted the system, but BOOM! It won\u2019t boot.<\/p>\n\n\n\n
The BIOS knows the bootable drive, but when I tried to boot it, it just went back into the BIOS. It didn\u2019t even give me a GRUB prompt or menu.<\/p>\n\n\n\n
I figured this wasn\u2019t a big deal, so I booted up the system with the 24.04 LTS Live USB. The plan is to just reinstall GRUB, and hopefully, that will fix the system.<\/p>\n\n\n\n
Once I\u2019ve booted into the Live USB and picked English as my language, I can jump into a command shell by pressing ALT-F2. Alternatively, you can press F1 and choose the shell option from the help menu. But, I found that the first method opens up a shell with command line completion, so I went with that.<\/p>\n\n\n\n
The boot disk had the following layout (output from both fdisk<\/code> and parted<\/code>):<\/p>\n\n\n\nsudo fdisk -l \/dev\/nvme1n1\nDisk \/dev\/nvme1n1: 931.51 GiB, 1000204886016 bytes, 1953525168 sectors\nDisk model: Samsung SSD 980 PRO 1TB\nUnits: sectors of 1 * 512 = 512 bytes\nSector size (logical\/physical): 512 bytes \/ 512 bytes\nI\/O size (minimum\/optimal): 512 bytes \/ 512 bytes\nDisklabel type: gpt\nDisk identifier: 90B9F208-2D05-484D-8C8C-B3AE71475167\n\nDevice Start End Sectors Size Type\n\/dev\/nvme1n1p1 2048 2203647 2201600 1G EFI System\n\/dev\/nvme1n1p2 2203648 1921875000 1919671353 915.4G Linux filesystem\n\/dev\/nvme1n1p3 1921875968 1953523711 31647744 15.1G Linux swap\n\nsudo parted \/dev\/nvme1n1 \nGNU Parted 3.4\nUsing \/dev\/nvme1n1\nWelcome to GNU Parted! Type 'help' to view a list of commands.\n(parted) p\nModel: Samsung SSD 980 PRO 1TB (nvme)\nDisk \/dev\/nvme1n1: 1000GB\nSector size (logical\/physical): 512B\/512B\nPartition Table: gpt\nDisk Flags:\n\nNumber Start End Size File system Name Flags\n 1 1049kB 1128MB 1127MB fat32 boot, esp\n 2 1128MB 984GB 983GB ext4\n 3 984GB 1000GB 16.2GB linux-swap(v1) swap swap<\/code><\/pre>\n\n\n\nAs I described in this post<\/a>, we want to make sure that the first partition is marked for EFI boot. This can be done in parted<\/code> with:<\/p>\n\n\n\nset 1 boot on\nset 1 esp on<\/code><\/pre>\n\n\n\nI didn’t have to perform the above since the first partition (\/dev\/nvme1n1p1<\/code>) is already recognized as EFI System<\/code>. We also need to ensure that this partition is formatted with FAT32. This can be done with:<\/p>\n\n\n\nsudo mkfs.vfat -F 32 \/dev\/nvme1n1p1<\/code><\/pre>\n\n\n\nSince this was already the case, I also did not have to perform this formatting step.<\/p>\n\n\n\n
The next step is to mount the root directory and the boot partition.<\/p>\n\n\n\n
mount \/dev\/nvme1n1p2 \/mnt\nmount \/dev\/nvme1n1p1 \/mnt\/boot\/efi<\/code><\/pre>\n\n\n\nWe now need to bind certain directories under \/mnt<\/code> in preparation for us to change our root directory to \/mnt<\/code>.<\/p>\n\n\n\nfor i in \/dev \/dev\/pts \/proc \/run; do sudo mount --bind $i \/mnt$i; done\nmount --rbind \/dev \/mnt\/dev\nmount --rbind \/sys \/mnt\/sys\nmount --rbind \/run \/mnt\/run\nmount -t proc \/proc \/mnt\/proc\nchroot \/mnt\ngrub-install --efi-directory=\/boot\/efi \/dev\/nvme1n1\nupdate-grub\n\nmount --make-rslave \/mnt\/dev\numount -R \/mnt\nexit<\/code><\/pre>\n\n\n\nIf we do not use the –rbind option for \/sys, then we may get an EFI error when running grub-install. There are two alternatives that solves the same issue, although used less often, you can also choose one of the following (but not BOTH):<\/p>\n\n\n\n
mount --bind \/sys\/firmware\/efi\/efivars \/mnt\/sys\/firmware\/efi\/efivars\nmount -t efivarfs none \/sys\/firmware\/efi\/efivars<\/code><\/pre>\n\n\n\nThe reinstallation of GRUB did not solve the problem. I had to perform a full system restore using my backup. The backup was created using rsync as described on this post<\/a>. However, I learned that this backup was done incorrectly! I excluded certain directories using the name<\/code> instead of \/name<\/code>. This caused more exclusion than intended. The correct method of the backup should be:<\/p>\n\n\n\nsudo rsync --delete \\\n --exclude '\/dev' \\\n --exclude '\/proc' \\\n --exclude '\/sys' \\\n --exclude '\/tmp' \\\n --exclude '\/run' \\\n --exclude '\/mnt' \\\n --exclude '\/media' \\\n --exclude '\/cdrom' \\\n --exclude 'lost+found' \\\n -aAXv \/ ${BACKUP}<\/code><\/pre>\n\n\n\nand the restoration command is very similar:<\/p>\n\n\n\n
mount \/dev\/sdt1 \/mnt\/backup\nmount \/dev\/nvme1n1p2 \/mnt\/system\n\nsudo rsync --delete \\\n --exclude '\/dev' \\\n --exclude '\/proc' \\\n --exclude '\/sys' \\\n --exclude '\/tmp' \\\n --exclude '\/run' \\\n --exclude '\/mnt' \\\n --exclude '\/media' \\\n --exclude '\/cdrom' \\\n --exclude 'lost+found' \\\n -aAXv \/mnt\/backup\/ \/mnt\/system\/<\/code><\/pre>\n\n\n\nAfter the restore, double check that \/var\/run<\/code> is soft-linked to \/run<\/code>.<\/p>\n\n\n\nOnce the restoration is completed, I follow the above instructions again to re-install GRUB, and I was able to boot back into my boot disk.<\/p>\n\n\n\n
Since this upgrade attempt has failed, I now have to figure out a way to move my system forward. I think what I will do is to port all of my services on my NAS as podman root-less quadlets<\/code>, and then just move the services into a brand new Ubuntu clean installation. This is probably easier to manage in the future.<\/p>\n","protected":false},"excerpt":{"rendered":"Last Saturday, I decided it was time to switch my NAS server from 22.04 LTS to 24.04 LTS. I\u2019ve been putting it off for ages, worried that the upgrade might not go as planned and something could go wrong. Since 24.04 is already in its fourth point release, I figured the risks should be manageable … Continue reading “Ubuntu 22.04 LTS to 24.04 LTS Upgrade Fail”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[111],"tags":[97,5,6],"class_list":["post-3327","post","type-post","status-publish","format-standard","hentry","category-tech","tag-linux","tag-nas","tag-ubuntu"],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p7V6i8-RF","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/blog.lufamily.ca\/kang\/wp-json\/wp\/v2\/posts\/3327","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.lufamily.ca\/kang\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.lufamily.ca\/kang\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.lufamily.ca\/kang\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.lufamily.ca\/kang\/wp-json\/wp\/v2\/comments?post=3327"}],"version-history":[{"count":7,"href":"https:\/\/blog.lufamily.ca\/kang\/wp-json\/wp\/v2\/posts\/3327\/revisions"}],"predecessor-version":[{"id":3335,"href":"https:\/\/blog.lufamily.ca\/kang\/wp-json\/wp\/v2\/posts\/3327\/revisions\/3335"}],"wp:attachment":[{"href":"https:\/\/blog.lufamily.ca\/kang\/wp-json\/wp\/v2\/media?parent=3327"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.lufamily.ca\/kang\/wp-json\/wp\/v2\/categories?post=3327"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.lufamily.ca\/kang\/wp-json\/wp\/v2\/tags?post=3327"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}