• * The Ubuntu/Fedora rootfs on tftp/nfs server:
  It can refer to the my article MIPS/ARM64 Debian/Ubuntu debootstrap for building the Ubuntu rootfs.
  If want to build the Fedora rootfs, it can install a new system on qemu and copy it from the qcow2 image.

• * PXE Server:
  You can refer my article ARM64 Centos8/Ubuntu PXE Boot Install on Ubuntu Server to build a PXE server.

• NFS Server:
  Assume the rootfs for Ubuntu is on /nfs/ubuntu2204 and Fedora 36 is on /nfs/fedora36.

Ubuntu PXE Boot

Copy vmlinux and initramfs to tftp server. Copy your current system’s vmlinux to your tftpboot. For example, create a directory for Ubuntu 22.04 in /tftpboot/nfs/ubuntu2204, and copy the vmlinuz to the folder using the following command:

$ cp /boot/vmlinuz-$(uname -r) /tftpboot/nfs/ubuntu2204/vmlinuz

Copy the initrd.img to the folder using the following command:

$ cp /boot/initrd.img-$(uname -r) /tftpboot/nfs/ubuntu2204/initrd.img

Setup your PXE server’s grub.cfg and add a record. Use the following example for the record:

menuentry 'Ubuntu22.04 nfsboot' { 
    linux nfs/ubuntu2204/vmlinuz ip=dhcp root=/dev/nfs nfsroot=<NFS server IP>:/nfs/ubuntu2204,nolock rw netboot=nfs 
    initrd nfs/ubuntu2204/initrd.img 
}

Fedora PXE Boot

The procedure is similar to Ubuntu, but the default Fedora initramfs doesn’t support nfs boot.
1. Rebuild the initramfs with dracut. You need to add nfs and network support to your Fedora initramfs using the following command:

$ dracut --add nfs --add network --force

It will generate the same name vmlinuz and initramfs. Copy it to your directory just like Ubuntu.
2. Disable selinux.
Default Fedora selinux is enabled, but it cannot enable on nfs rootfs (or it can but needs to set up something). You can disable it by adding selinux=0 on boot kernel parameter. Use the following example for the record:

linux nfs/fedora36/vmlinuz boot=nfs root=/dev/nfs nfsroot=<NFS server IP>:/nfs/fedora36,nolock rw netboot=nfs ip=dhcp selitarnux=0

The post Ubuntu/Fedora PXE Boot appeared first on richliu's blog.

Hardware topology 

[ARM64 CLIENT] <——-> [Server]  <——-> Internet 

SERVER ethernet ip is 10.0.0.1 , another interface connect to internet. 

Install necessary packages 

Install TFTP, HTTP and DHCPD Server and rpm2cpio 

$ apt install tftpd-hpa apache2 isc-dhcp-server rpm2cpio

Modify dhcpd server configuration 

$ vim /etc/dhcp/dhcpd.conf

following is example configuration,it can be changed to fit your environment 

authoritative;
default-lease-time                      259200;
max-lease-time                          518400;
option routers                          10.0.0.1;
option subnet-mask                      255.255.255.0;
option domain-name-servers              168.95.1.1,8.8.8.8;
ddns-update-style                       interim;
ignore client-updates;
allow booting;
allow bootp;
allow unknown-clients;
server-name                             pxis;

subnet 10.0.0.0 netmask 255.255.255.0 {
    range 10.0.0.100 10.0.0.200;
    option ip-forwarding off;
    option subnet-mask                  255.255.255.0;
    next-server 10.0.0.1;
    filename "shim.efi";

}

Modify tftpd setting 

$ vim /etc/default/tftpd-hpa

Add –create to TFTP_OPTIONS and change tftp directory to /tftpboot 

following is example configuration file 

TFTP_USERNAME="tftp"
TFTP_DIRECTORY="/tftpboot"
TFTP_ADDRESS=":69"
TFTP_OPTIONS="--secure  --create"

Restart Service 

mkdir /tftpboot 
/etc/init.d/tftpd-hpa restart
/etc/init.d/apache2 restart
/etc/init.d/isc-dhcp-server restart

Get ARM64 pxeboot file 

some times the shim and grub2-efi would be erase, just visit mirror.centos.org and find new one than download it.

cd /tftpboot 
wget http://mirror.centos.org/centos/8-stream/BaseOS/aarch64/os/Packages/shim-aa64-15-15.el8_2.aarch64.rpm
wget http://mirror.centos.org/centos/8-stream/BaseOS/aarch64/os/Packages/grub2-efi-aa64-2.02-106.el8.aarch64.rpm
rpm2cpio shim-aa64-15-15.el8_2.aarch64.rpm | cpio -dimv
rpm2cpio grub2-efi-aa64-2.02-106.el8.aarch64.rpm | cpio -dimv
cp ./boot/efi/EFI/centos/shim.efi .
cp ./boot/efi/EFI/centos/grubaa64.efi .
chmod +rx *.efi

Configure grub.conf 

$ vim /tftpboot/grub.cfg

Following is grub.conf example 

set timeout=60
menuentry 'CentOS' {
  linux images/pxeboot/vmlinuz ip=dhcp inst.repo=http://10.0.0.1/centos8/
  initrd images/pxeboot/initrd.img
}

If your system support secure boot and you want to do it, please replace linux with linuxefi and replace initrd with initrdefi. 

Get CentOS 8 stream image and copy it to web server 

cd /var/www/html 
wget http://ftp.twaren.net/Linux/CentOS/8-stream/isos/x86_64/CentOS-Stream-8-x86_64-20220215-dvd1.iso
mkdir disk 
mount -o loop CentOS-Stream-8-x86_64-20220215-dvd1.iso disk
cp -a disk centos8 
umount disk 
rmdir disk

# copy boot file into tftp directory 

cd /tftpboot
cp /var/www/html/centos8/images/ . -a

After copy all file into http file directory, it can visit via browser on server with URL http://10.0.0.1/centos8/ to check whether file exist or not. 

NAT

Server should support NAT function to forward packet. 

assume interface connect to internet is wls3
Following is a sample command to enable Linux NAT function which can make client to access internet or intranet . 

$ echo 1 > /proc/sys/net/ipv4/ip_forward ==> enable it immediately 
$ iptables -t nat -A POSTROUTING -o wls3 -j MASQUERADE

Now, you can choose pxeboot on your ARM64 machine. if everything successful, you can see CentOS on your console if everything is ok . 

Known Issue

Stdin: Invalid argument

My system cannot boot into the Ubuntu install program for an unknown reason. After checking the system, it seems that the issue may be caused by an inability to access internet resources for some reason, even though the DNS and network appear to be fine. An alternative solution is to put the ISO on a website and assign the argument to the parameter. Here is an example using Ubuntu 22.04.

menuentry 'Ubuntu22.04 Net Install' {
linux ubuntu2204/vmlinuz ip=dhcp url=http://10.0.0.1/ubuntu-22.04.1-live-server-arm64.iso only-ubiquity
initrd ubuntu2204/initrd
}

Debug 

Wireshark is your good friend, use it. 

Ubuntu PXE Environment

It can download Ubuntu Netboot images from this url.
http://cdimage.ubuntu.com/netboot/

Choice what you want, for example, url below is Ubuntu 18.04 for ARM64 image
http://ports.ubuntu.com/ubuntu-ports/dists/bionic-updates/main/installer-arm64/current/images/netboot/
Ubuntu 20.04 Netboot file
http://ports.ubuntu.com/ubuntu-ports/dists/focal-updates/main/installer-arm64/current/legacy-images/netboot/netboot.tar.gz

Download netboot image to tftp directory, I prefer to put different version to different directory.

$ cd /tftpboot
$ wget http://ports.ubuntu.com/ubuntu-ports/dists/bionic-updates/main/installer-arm64/current/images/netboot/netboot.tar.gz
$ mkdir ubuntu1804
$ cd ubuntu1804
$ tar xvf ../netboot.tar.gz

Modify grub.cfg and add Ubuntu option to grub.cfg, save it, now, this server can also provide ubuntu net install

set timeout=60
menuentry 'CentOS8 Install' {
  linux images/pxeboot/vmlinuz ip=dhcp inst.repo=http://10.0.0.1/centos8/
  initrd images/pxeboot/initrd.img
}
menuentry 'Ubuntu18.04 Install' {
  linux ubuntu1804/ubuntu-installer/arm64/linux ip=dhcp
  initrd ubuntu1804/ubuntu-installer/arm64/initrd.gz
}

Ubuntu ISO environment

It can download iso file and use the iso file as install media.

in Linux, download Ubuntu 20.04 ARM64 iso file

$ wget https://cdimage.ubuntu.com/releases/20.04/release/ubuntu-20.04.3-live-server-arm64.iso
$ mount -o loop ubuntu-20.04.3-live-server-arm64.iso disk
$ mkdir ubuntu2004
$ cp disk/casper ubuntu2004/ -a 
$ umount disk

Modify grub.cfg as below (not test)

set timeout=60
menuentry 'CentOS8 Install' {
  linux images/pxeboot/vmlinuz ip=dhcp inst.repo=http://10.0.0.1/centos8/
  initrd images/pxeboot/initrd.img
}
menuentry 'Ubuntu20.04 Install' {
  linux ubuntu2004/casper/vmlinuz ip=dhcp
  initrd ubuntu2004/casper/initrd
}

It also can use HWE kernel

menuentry 'Ubuntu20.04 Install' {
  linux ubuntu2004/casper/hwe-vmlinux ip=dhcp
  initrd ubuntu2004/casper/hwe-initrd
}

Note for GRUB Install Command

CentOS

svr=10.0.0.1; root=boot/centos8/images/pxeboot
linux (http,$svr)/$root/vmlinuz inst.stage2=http://$svr/boot/centos8 text
initrd (http,$svr)/$root/initrd.img

Ubuntu

svr=192.168.110.254; root=boot/ubuntu2004/casper
iso=ubuntu-20.04.3-live-server-arm64.iso
linux (http,$svr)/$root/vmlinuz url=http://192.168.110.254/boot/$iso only-ubiquity ip=dhcp ---
initrd (http,$svr)/$root/initrd

linux (http,$svr)/$root/hwe-vmlinuz url=http://192.168.110.254/boot/$iso only-ubiquity ip=dhcp ---
initrd (http,$svr)/$root/hwe-initrd

Fedora 36

Fedora’s setting is more closed to CentOS, due to CentOS might be dropped recently, so, moved to Fedora environment is necessary.

Get initrd/vmlinux image

Other parts like CentOS,
Download the PXE initrd and vmlinuz image, free.nchc.org.tw is Taiwan local Fedora server, you can replace it with local server.

mkdir -p /tftpboot/f36

cd /tftpboot/f36
wget http://free.nchc.org.tw/fedora/linux/releases/36/Server/aarch64/os/images/pxeboot/initrd.img
wget http://free.nchc.org.tw/fedora/linux/releases/36/Server/aarch64/os/images/pxeboot/vmlinuz

cd /var/www/html
wget http://free.nchc.org.tw/fedora/linux/releases/36/Server/aarch64/iso/Fedora-Server-dvd-aarch64-36-1.5.iso
mkdir disk 
mount -o loop Fedora-Server-dvd-aarch64-36-1.5.iso disk
cp -a disk f36
umount disk

Grub Fedora Setting

Add following grub entry text into /tftpboot/grub.cfg, thus, it can insert a “Fedora36” menu on pxe boot.

menuentry 'Fedora36' {
  linux f36/vmlinuz ip=dhcp inst.repo=http://10.0.0.1/f36/
  initrd f36/initrd.img
}

Ref. 

https://docs.centos.org/en-US/centos/install-guide/pxe-server/#sect-network-boot-setup-uefi
https://docs.centos.org/en-US/8-docs/advanced-install/assembly_preparing-for-a-network-install/
https://forums.opensuse.org/showthread.php/501075-error-cannot-find-commands-linuxefi-or-initrdefi

The post ARM64 Centos8/Ubuntu PXE Boot Install on Ubuntu Server appeared first on richliu's blog.