This is a work note for run x86_64/AM64 on ARM64/AARCH64 server .
Because they are different archeticture, so, run x86_64 on ARM server would be very slow.
and some functions might be different when run on different platform
Ubuntu
It can refer to those three articles to create a new cloud image from Ubuntu server image, remember to change ARM64 image to x86_64 image .
Change Ubuntu Cloud Image Size (if you choice ubuntu 16.04 as your image)
Change Ubuntu Cloud Image Password
Ubuntu Cloud Image Taiwan Mirror Site
Install Qemu and setup bridge devices
apt install -y ovmf qemu bridge-utils
mkdir -p /etc/qemu
echo "allow br0" > /etc/qemu/bridge.conf
echo "allow virbr0" >> /etc/qemu/bridge.confWrite this script to a file and run it.
SERVERFILE=xenial-server-cloudimg-amd64-uefi1.img
VERSION=`uname -r`
sudo qemu-system-x86_64 -name vm1 \
-machine pc-i440fx-xenial,usb=off \
-cpu qemu64 -m 8192 \
-smp 12,sockets=1,cores=12,threads=1 \
-nographic -nodefaults \
-bios /usr/share/ovmf/OVMF.fd \
-virtfs local,path=/home/work,mount_tag=host0,security_model=passthrough,id=host0 \
-drive file=$SERVERFILE \
-net nic,macaddr=$macaddr \
-net tap,ifname=tap0 \
-serial telnet::9001,server,nowait > guest1_log.txt 2>&1 &
sleep 5
brctl addif br0 tap0it can use command
qemu-system-x86_64 –machine help
and
qemu-system-x86_64 –cpu help
to get all machine and cpu options.
it can change cpu and machine type here.
network would be bridge on tap0
below is good option to share folder between host and VM, below command will share /home/work with vm.
option
--virtfs local,path=/home/work,mount_tag=host0,security_model=passthrough,id=host0 it can put this line to /etc/fstab to mount host’s folder to somewhere, in this case, just put the same directory like host does.
host0 /home/work 9p trans=virtio,version=9p2000.L,user,umask=000 0 0CentOS
CentOS document was written on 2021, but Ubuntu document on 2019, there is some difference between it. but that’s ok, I believe both ways are works.
Compile QEMU
CentOS default application doesn’t support run x86_64 code on ARM64 server, it needs to compile manually.
Please download qemu and compile it.
# Install CentOS Packages
yum config-manager --set-enabled powertools
yum install -y xorg-x11-xauth python3 ninja-build
yum install -y glib2-devel libmount-devel
yum install -y git glib2-devel libfdt-devel pixman-devel zlib-devel
# Download QEMU Source Code
wget https://download.qemu.org/qemu-5.2.0.tar.xz
# Decompress it
tar xvf qemu-5.2.0.tar.xz
#
cd qemu-5.2.0
./configure --target-list="x86_64-softmmu aarch64-softmmu aarch64-linux-user x86_64-linux-user"
make -j `nproc`
make install
Then, qemu will be installed on /usr/local/bin
Download UEFI Image
It needs UEFI Image for x86_64 image boot up. It needs OVMF’s tianocore (Open Source UEFI BIOS) to boot up. And we can download kraxel as prebuild images.
# Install Kraxel into CentOS repos
cd /etc/yum.repos.d/
wget https://www.kraxel.org/repos/firmware.repo
yum install edk2.git-aarch64 edk2.git-ovmf-x64
cd <your vm directory>
cp /usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd .
cp /usr/share/edk2.git/ovmf-x64/OVMF_VARS-pure-efi.fd .
Prepare Bridge Devices
It needs bridge devices to connect to internet, default qemu bridge device name is virbr0 , due to CentOS doesn’t support brctl, follow this document(3 Ways to Create a Network Bridge in RHEL/CentOS 8) to create a CentOS bridge devices. Below is my experiment script to create bridge device, this setting will keep in system, reboot will not be reset.
nmcli conn add type bridge con-name virbr0 ifname virbr0
killall dhclient
ip addr flush enP7p2s0
nmcli conn modify virbr0 ipv4.addresses '192.168.3.191/16'
nmcli conn modify br0 ipv4.gateway '192.168.1.1'
nmcli conn modify br0 ipv4.dns '8.8.8.8'
nmcli conn modify br0 ipv4.method manual
nmcli conn add type ethernet slave-type bridge con-name bridge-br0 ifname enP7p2s0 master virbr0
nmcli conn up virbr0
nmcli conn up bridge-br0
nmcli conn show --active
# Allow QEMU to access virbr0
mkdir -p /etc/qemu
echo "allow virbr0" >> /etc/qemu/bridge.conf
Run Qemu
It can refer above to prepare cloud image, I prefer to use ubuntu cloud image, but have not try CentOS image.
Remember Ubuntu 18.04 image won’t auto increase image size, please follow this Change Ubuntu Cloud Image Size to modify image size
qemu-system-x86_64 -m 8192 -smp 16 -nographic \
-drive if=pflash,format=raw,unit=0,file=OVMF_CODE-pure-efi.fd,readonly=on \
-drive if=pflash,format=raw,unit=1,file=OVMF_VARS-pure-efi.fd \
-hdc bionic-server-cloudimg-amd64.img \
-netdev bridge,br=virbr0,id=net0 \
-device virtio-net,netdev=net0,mac=00:11:22:33:44:55 \
-serial telnet::9001,server,nowait > vmlog.txt 2>> vmlog.txt &
x86_64 application
Sometimes we don’t need to run whole VM to run some application just for x86_64, it can run user mode application directly. I am a lazy guy, so just reuse cloud image as x86_64 Linux library.
mkdir -p /home/amd64
qemu-img convert bionic-server-cloudimg-amd64.img bionic-server-cloudimg-amd64.raw
losetup /dev/loop0 bionic-server-cloudimg-amd64.raw
kpartx -a /dev/loop0
mount /dev/mapper/loop0p1 /home/amd64
Umount Image
umount /home/amd64
losetup -d /dev/loop0Run x86_64 only application
qemu-x86_64 -L /home/amd64/ bootutil64e
qemu-x86_64 -L /home/amd64/ nvmupdate64e -lUnsolved Problem
Cannot find way to update INTEL NIC’s NVM
Tried vfio and application mode, vfio cannot pass iommu to host, it will fail.
User space application will cause mmap fail.
If you know how to update Intel NIC’s NVM, welcome to leave a message for me.
QEMU –static problem
Add –static parameter will cause some problem, cannot solve it right now.
./configure –target-list=”x86_64-softmmu aarch64-softmmu aarch64-linux-user x86_64-linux-user” –static
===
ERROR: sizeof(size_t) doesn’t match GLIB_SIZEOF_SIZE_T.
You probably need to set PKG_CONFIG_LIBDIR
to point to the right pkg-config files for your build target
發佈留言