Installing FreeBSD 14 on Raspberry Pi 4B with ZFS root

None

Yes, it won't be the most performative, but we're just looking at DHCP and DNS, so shouldn't require much.

I had a spare 250GB SATA SSD drive laying around and picked up a SATA->USB3 adaptor.  Testing showed this was at least 5 times faster than using a microSD card on the Pi.

All the FreeBSD installation instructions for the Pi just show writing (dd) the image file to the boot drive, but that will give you a ufs partition and my goal is to run zfs as the root on the SSD drive (and not need a SD card at all).

I also found a number of other people on the internet doing the same thing, but none of those insturctions were fully complete or exactly what I wanted, so I'm documenting everything I did here for my own future reference.

Most of these instructions are based off of the following reddit post, which was the best example I found:
https://www.reddit.com/r/freebsd/comments/v4b5oj/freebsd_on_zfs_on_raspberry_pi/

FreeBSD Images for the Pi found here: https://download.freebsd.org/releases/arm64/aarch64/ISO-IMAGES/14.0/ (the one that ends in -RPI.img.xz)

Step 1: Install FreeBSD on an SD card

This was just following the default instructions found here: https://wiki.freebsd.org/arm/Raspberry%20Pi

I used the Raspberry Pi Imager to write the image to the SD card from another computer.

Step 2: Boot the Pi from the SD card, then attach the USB SSD drive

The rest of the instructions are done after SSHing into the Pi and running as root.

Step 3: Prepare the SSD drive

Download the Installer Image to the Pi (sdcard):

curl -L -O https://download.freebsd.org/releases/arm64/aarch64/ISO-IMAGES/14.0/FreeBSD-14.0-RELEASE-arm64-aarch64-RPI.img.xz
unxz FreeBSD-14.0-RELEASE-arm64-aarch64-RPI.img.xz

Write the image to the target disk (in my case, da0).  This will create the partitions, a fat32 boot partition(1) and a ufs partition (2) with freebsd.  We'll blow away this 2nd partition and replace it with our zpool, but this step gets things bootable.

dd if=FreeBSD-14.0-RELEASE-arm64-aarch64-RPI.img  of=/dev/da0 bs=4M iflag=direct oflag=direct status=progress
gpart resize -i 2 da0
gpart resize -i 1 da0s2

Next, replace the ufs partition just created (da0s2) with a zpool.

zpool create -O compress=lz4 -O atime=off -o altroot=/zpialt -m none zroot da0s2

zfs create -o mountpoint=none zroot/ROOT
zfs create -o mountpoint=/ zroot/ROOT/default
zfs create -o mountpoint=/tmp -o exec=on -o setuid=off zroot/tmp
zfs create -o mountpoint=/usr -o canmount=off zroot/usr
zfs create zroot/usr/home
zfs create -o setuid=off zroot/usr/ports
zfs create zroot/usr/src
zfs create -o mountpoint=/var -o canmount=off zroot/var
zfs create -o exec=off -o setuid=off zroot/var/audit
zfs create -o exec=off -o setuid=off zroot/var/crash
zfs create -o exec=off -o setuid=off zroot/var/log
zfs create -o atime=on zroot/var/mail
zfs create -o setuid=off zroot/var/tmp

zfs set mountpoint=/zroot zroot

zpool set bootfs=zroot/ROOT/default zroot

zfs set canmount=noauto zroot/ROOT/default

Create a loopback device (md0) and mount that installer .img previously downloaded.

mdconfig -a -t vnode -f FreeBSD-14.0-RELEASE-arm64-aarch64-RPI.img -u 0
mount /dev/md0s2a /mnt

And copy the installation files into the new zfs filesystem (that's been mounted under /zpialt)

cd /mnt
tar cf - . | ( cd /zpialt && tar xvf - )

Step 4: Cleanup

Before rebooting into the new zfs root install, I made sure to make a few little changes to the install.

echo 'zfs_load="YES"' >> /zpialt/boot/loader.conf
echo 'kern.geom.label.disk_ident.enable=0' >> /zpialt/boot/loader.conf
echo 'kern.geom.label.gptid.enable=0' >> /zpialt/boot/loader.conf
echo 'net.fibs=5' >> /zpialt/boot/loader.conf

sed -i '' '/boot_serial/ s/^/#/' /zpialt/boot/loader.conf

sed -i '' /ufs/d /zpialt/etc/fstab
 

Edit the /zpialt/etc/rc.conf.  Since this Pi will be running DHCP, I need to set a static IP, but YMMV.

hostname="digitama.int.axcella.net"
ifconfig_genet0="inet 10.0.80.13 netmask 255.255.255.0"
defaultrouter="10.0.80.1"

vlans_genet0="10 11 12 13"
ifconfig_genet0_10="inet 10.0.10.4/24"
ifconfig_genet0_11="inet 10.0.11.4/24"
ifconfig_genet0_12="inet 10.0.12.4/24"
ifconfig_genet0_13="inet 10.0.13.4/24"

sshd_enable="YES"
sendmail_enable="NONE"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"

ntpdate_enable="YES"
ntpdate_hosts="10.0.80.1"

# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="NO"
nfs_reserved_port_only="NO"
powerd_enable="YES"
zfs_enable="YES"

Add a /etc/resolv.conf:

search int.axcella.net
#nameserver 127.0.0.1
nameserver 10.0.80.1

And finally

zpool export zroot

Step 5: Reboot

Shutdown, remove the sdcard and reboot into newly install disk.

 

You Might Also Like