Encrypted root disk migration for FreeBSD
I’ve had a VPS with ARP Networks for a long time now. Things were a bit different back then. The default FreeBSD installer suggested setting up multiple partitions (slices) on a disk by default. This is no longer the case. Encryption wasn’t a thing that people generally worried about. I’ve had it on my todo list for a while now to figure out how to converge to a single encrypted partition on my VPS — saving me from running out of space in /var/, and also to protect the data on the underlying “disk”. I finally worked it out a few weeks ago.
The helpful folks in the #arpnetworks IRC channel on Freenode pointed me in the right direction. A tutorial on full disk encryption got me started. However, this was only helpful on a new install. My VPS is over 10 years old, and I didn’t really want to set everything up from scratch. Also helpful was FreeBSD’s dump/restore combined with an additional disk attached to my VPS.
To start, I backed up all my existing partitions to an additional disk:
# dump -C16 -b64 -0aL -h0 -f /mnt/root.dump /
# dump -C16 -b64 -0aL -h0 -f /mnt/tmp.dump /tmp
# dump -C16 -b64 -0aL -h0 -f /mnt/var.dump /var
# dump -C16 -b64 -0aL -h0 -f /mnt/usr.dump /usr
Note that I backed up the partitions in order of volatility, figuring /usr would have the most changes by the time I finished the dump process. The -L option made it possible to dump a mounted filesystem.
Once the dump was complete, I booted from a FreeBSD install CD. When the installer popped up, I chose to start a shell.
Because I already had a had an existing partition table, I first had to destroy it before creating a new one:
# gpart destroy -F da0
# gpart create -s gpt da0
I laid out my partition table with the unencrypted boot partition first, then a swap partition, and then my root partition. To do that, I used the following commands:
# gpart add -t freebsd-boot -s 512k -a 4k da0
# gpart add -t freebsd-ufs -l bootfs -s 1g -a 1m da0
# gpart add -t freebsd-ufs -l bootfs -s 2g -a 1m da0
# gpart add -t freebsd-ufs -l encrypted -a 1m da0
I then installed the bootcode:
# gpart bootcode -b /boot/pmbr -p /boot/gptboot -i 1 da0
Initializing geli was easy:
# geli init -b -s 4096 da0p4
Once initialized, the device must be attached, which creates /dev/da0p4.eli:
# geli attach da0p4
Now the partitions can be formatted:
# newfs -U /dev/da0p2
# newfs -U /dev/da0p4.eli
With everything formatted, I then mounted /dev/da0p4.eli to /mnt as this is where all subsequent work would be performed:
# mount /dev/da0p4.eli /mnt
Because my backups were on /dev/da1, I had to mount them somewhere to perform the restore. I created a new directory in /mnt/ (/mnt/mnt) and mounted /dev/da1s1 there:
# mkdir /mnt/mnt
# mount /dev/da1s1 /mnt/mnt
Now I was ready to restore the filesystems atop each other. I wasn’t sure if this would work, but it did!
# cd /mnt
# restore -ruf /mnt/root.dump
# rm restoresymtable
# cd /mnt/tmp
# restore -ruf /mnt/tmp.dump
# rm restoresymtable
# cd /mnt/var
# restore -ruf /mnt/var.dump
# rm restoresymtable
# cd /mnt/usr
# restore -ruf /mnt/usr.dum
# rm restoresymtable
Note that the dump will be restore to the current directory, hence cd before each restore.
This restored /boot to my encrypted partition, which isn’t what I wanted. I had to remove the restored /boot and then set up the unencrypted partition and restore again. I discovered the -x option for restore which let me restore only the /boot directory to the unencrypted partition.
# rm -rf /mnt/boot
# mkdir /mnt/unenc
# mount /dev/ada0p2 /mnt/unenc
# cd /mnt/unenc
# restore -xuf /mnt/root.dump boot
You have not read any tapes yet.
If you are extracting just a few files, start with the last volume
and work towards the first; restore can quickly skip tapes that
have no further files to extract. Otherwise, begin with volume 1.
Specify next volume #: 1
set owner/mode for '.'? [yn] y
With everything in place, I could then add the symlink for /boot to the unencrypted partition:
# cd /mnt
# ln -s unenc/boot /mnt/boot
Having restored the backups, I then had to set up my /etc/fstab and /boot/loader.conf. I replaced /mnt/etc/fstab with the following:
# Device Mountpoint FStype Options Dump Pass#
/dev/da0p3.eli none swap sw 0 0
/dev/da0p2 /unenc ufs rw 1 1
/dev/da0p4.eli / ufs rw 2 2
Note that while I did not geli init or geli attach /dev/da0p3, /sbin/swapon will detect the .eli suffix and encrypt the partition with a one-time key.
The FreeBSD bootloader must be configured to load geli and informed of where the encrypted root partition. I added the following to /mnt/unenc/boot/loader.conf:
geom_eli_load="YES"
vfs.root.mountfrom="ufs:ada0p3.eli"
Note that if you have a CPU with AESNI (ARP Networks VPSes do) then you should also add aesni_load="YES" to loader.conf for increased performance.
With everything in place I rebooted and was prompted to enter my password:
Enter passphrase for da0p4:
I was truly impressed at how easy this all was, especially with dump and restore, which I’d never used before! After finishing the migration, I discovered that geli init option -g which supposedly obviates the need for the unencrypted /boot partition. I’ve not tried this out myself, but perhaps that’s something to explore on a rainy day.