[root@localhost linux-4.18.0-80.11.2.el8_0]# make install                                            [55/2101]
/bin/sh ./arch/arm64/boot/install.sh 4.18.0 \
arch/arm64/boot/Image System.map "/boot"
dkms: running auto installation service for kernel 4.18.0
Deprecated feature: REMAKE_INITRD
Deprecated feature: REMAKE_INITRD
Deprecated feature: REMAKE_INITRD
Sign command: /lib/modules/4.18.0/build/scripts/sign-file
Signing key: /var/lib/dkms/mok.key
Public certificate (MOK): /var/lib/dkms/mok.pub                                                               Deprecated feature: REMAKE_INITRD                                                                             
Building module:
Cleaning build area...(bad exit status: 2)
make -j160 KERNELRELEASE=4.18.0 -C /lib/modules/4.18.0/build M=/var/lib/dkms/bnxt_en/1.10.2.221.0.114.0/build
KVER=4.18.0...(bad exit status: 2)
Error! Bad return status for module build on kernel: 4.18.0 (aarch64)
Consult /var/lib/dkms/bnxt_en/1.10.2.221.0.114.0/build/make.log for more information.
Sign command: /lib/modules/4.18.0/build/scripts/sign-file
Signing key: /var/lib/dkms/mok.key
Public certificate (MOK): /var/lib/dkms/mok.pub
Deprecated feature: REMAKE_INITRD

Building module:
Cleaning build area...(bad exit status: 2)
make -j160 KERNELRELEASE=4.18.0 -C /lib/modules/4.18.0/build M=/var/lib/dkms/bnxt_re/221.0.114.0/build KVER=4$
18.0 BNXT_EN_INC=/var/lib/dkms/bnxt_re/221.0.114.0/source BNXT_QPLIB_INC=/var/lib/dkms/bnxt_re/221.0.114.0/so$
rce...(bad exit status: 2)
Error! Bad return status for module build on kernel: 4.18.0 (aarch64)
Consult /var/lib/dkms/bnxt_re/221.0.114.0/build/make.log for more information.
 Done.

And check the log file “/var/lib/dkms/bnxt_re/221.0.114.0/build/make.log”

It shows

DKMS make.log for bnxt_en-1.10.2.221.0.114.0 for kernel 4.18.0 (aarch64)
Tue Sep 27 15:41:04 CST 2022
Makefile:603: arch/aarch64/Makefile: No such file or directory
make[2]: *** No rule to make target 'arch/aarch64/Makefile'.  Stop.

Seems cannot find the Makefile, that because the drvier cannot locate right Makefile, because the include architecture is arm64, not aarch64 in Linux kernel .

By the way, basicly arm64 and aarch64 are the same thing, I don’t know why it shows aarch64 on kernel architecture and use arm64 as fold name. probably some traditional history.

solution is easy, go to the include folder and link arm64 to aarch, DONE! ex:

cd /lib/modules/4.18.0/build/arch
ln -sf arm64 aarch64

The post Linux Kernel Install Fail Because Broadcom NetXtreme 100G DKMS Driver Build Failed. appeared first on richliu's blog.

Install Necessary Packages

Install packages, it should have to install other packages like openssl-dev.

get package ampere-8.3.0-20191025-dynamic-nosysroot-nativetools.tar.xz from Ampere customer connect or ampere development source web.

yum groupinstall "Development Tools"

yum -y install ncurses-devel
yum -y install hmaccalc zlib-devel binutils-devel elfutils-devel libelf-devel newt-devel python-devel
yum -y install audit-libs-devel numactl-devel pciutils-devel openssl-devel
yum -y install bc bison perl-ExtUtils-Embed.noarch xmlto asciidoc
rpm –ivh kernel-4.18.0-80.11.2.el8.<version>.src.rpm


mkdir -p /opt/amp
tar -xf ampere-8.3.0-20191025-dynamic-nosysroot-nativetools.tar.xz -C /opt/amp
export PATH=/opt/amp/ampere-8.3.0-20191025-dynamic-nosysroot/bin:$PATH;
which gcc
/opt/amp/ampere-8.3.0-20191025/bin/gcc

if gcc is not in /opt/amp, please check your directory in /opt/amy

Build Kernel

command to build kernel

cd /root/rpmbuild/SOURCES/
tar xvf linux-4.18.0-80.11.2.el8_0.tar.xz
cd linux-4.18.0-80.11.2.el8_0
cp ../kernel-aarch64-emag.config .config
make olddefconfig

Try build once and make sure everything is ok

make -j `nproc`

If buiild OK, generate RPM files.

make -j `nproc` rpm-pkg

if everything is ok, the RPM files will be on /root/rpmbuild/RPMS/aarch64/

Install RPM packages

/root/rpmbuild/RPMS/aarch64
yum localinstall kernel-4.18.0-1.aarch64.rpm kernel-headers-4.18.0-1.aarch64.rpm

The post Build Ampere CentOS Kernel appeared first on richliu's blog.

Install CentOS 7 on Ampere Altra CPU

Earlier CentOS 7 default OS is too old to support ARM64 CPU and SSIF driver,
if you have problem to install on Ampre Altra CPU, it needs to block SSIF and change PCI parameter during booting. Add parameters below on Linux kernel booting parameter when install CentOS7.

modprobe.blacklist=ipmi_si pci=nomsi

Prepare Build Environment

Before download and build Linux kernel 5.x, it needs to run those command to prepare build environment

yum install -y centos-release-scl-rh
yum install -y devtoolset-9*
yum groupinstall -y "Development Tools"
yum install -y ncurses-devel
yum install -y hmaccalc zlib-devel binutils-devel elfutils-libelf-devel
yum install -y python3

Download Linux kernel

Ampere LTS kernel can be download from github, here is the command to checkout Ampere LTS kernel and switch version to 5.10

cd /root
git clone https://github.com/AmpereComputing/ampere-lts-kernel.git
cd /ampere-lte-kernel
git checkout -b origin/linux-5.10.y 5.10.y

On later Linux kernel, it needs to use git switch to switch to 5.10.y

git switch 5.10.y

Build Linux kernel

It’s simple to build Linux kernel

scl enable devtoolset-9 bash
cd /root/ampere-lte-kernel
cp arch/arm64/configs/altra_5.10_defconfig .config
make olddefconfig
make -j `nproc` && make -j `nproc` modules  && make modules_install && make install

GRUB Setting

Suggest to change default GRUB timeout to choice right kernel to boot.

sed -i 's/GRUB_TIMEOUT=5/GRUB_TIMEOUT=30/g' /etc/default/grub
grub2-mkconfig -o /boot/efi/EFI/centos/grub.cfg

OK, right now it can reboot your system to have try.

Ref:
Good mirror site for CentOS7 package.
https://mirrors.bfsu.edu.cn/centos-altarch/7.9.2009/os/aarch64/

The post Build Linux kernel 5.x on CentOS 7 appeared first on richliu's blog.

Ubuntu

It can refer to those three articles to create a new cloud image from Ubuntu server image, remember to change ARM64 image to x86_64 image .

Change Ubuntu Cloud Image Size (if you choice ubuntu 16.04 as your image)
Change Ubuntu Cloud Image Password
Ubuntu Cloud Image Taiwan Mirror Site

Install Qemu and setup bridge devices

apt install -y ovmf qemu bridge-utils
mkdir -p /etc/qemu
echo "allow br0" > /etc/qemu/bridge.conf
echo "allow virbr0" >> /etc/qemu/bridge.conf

Write this script to a file and run it.

SERVERFILE=xenial-server-cloudimg-amd64-uefi1.img
VERSION=`uname -r`

sudo qemu-system-x86_64 -name vm1 \
        -machine pc-i440fx-xenial,usb=off \
        -cpu qemu64 -m 8192 \
        -smp 12,sockets=1,cores=12,threads=1 \
        -nographic -nodefaults \
        -bios /usr/share/ovmf/OVMF.fd \
        -virtfs local,path=/home/work,mount_tag=host0,security_model=passthrough,id=host0 \
        -drive file=$SERVERFILE  \
        -net nic,macaddr=$macaddr \
        -net tap,ifname=tap0 \
        -serial telnet::9001,server,nowait > guest1_log.txt 2>&1 &

sleep 5

brctl addif br0 tap0

it can use command
qemu-system-x86_64 –machine help
and
qemu-system-x86_64 –cpu help
to get all machine and cpu options.

it can change cpu and machine type here.

network would be bridge on tap0

below is good option to share folder between host and VM, below command will share /home/work with vm.

option 
--virtfs local,path=/home/work,mount_tag=host0,security_model=passthrough,id=host0 

it can put this line to /etc/fstab to mount host’s folder to somewhere, in this case, just put the same directory like host does.

host0   /home/work   9p      trans=virtio,version=9p2000.L,user,umask=000   0 0

CentOS

CentOS document was written on 2021, but Ubuntu document on 2019, there is some difference between it. but that’s ok, I believe both ways are works.

Compile QEMU

CentOS default application doesn’t support run x86_64 code on ARM64 server, it needs to compile manually.

Please download qemu and compile it.

# Install CentOS Packages
yum config-manager --set-enabled powertools
yum install -y xorg-x11-xauth python3 ninja-build
yum install -y glib2-devel libmount-devel
yum install -y git glib2-devel libfdt-devel pixman-devel zlib-devel

# Download QEMU Source Code
wget https://download.qemu.org/qemu-5.2.0.tar.xz
# Decompress it
tar xvf qemu-5.2.0.tar.xz
# 
cd qemu-5.2.0
./configure --target-list="x86_64-softmmu aarch64-softmmu aarch64-linux-user x86_64-linux-user"
make -j `nproc`
make install 

Then, qemu will be installed on /usr/local/bin

Download UEFI Image

It needs UEFI Image for x86_64 image boot up. It needs OVMF’s tianocore (Open Source UEFI BIOS) to boot up. And we can download kraxel as prebuild images.

# Install Kraxel into CentOS repos
cd /etc/yum.repos.d/
wget https://www.kraxel.org/repos/firmware.repo
yum install edk2.git-aarch64 edk2.git-ovmf-x64
cd <your vm directory>
cp /usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd .
cp /usr/share/edk2.git/ovmf-x64/OVMF_VARS-pure-efi.fd .

Prepare Bridge Devices

It needs bridge devices to connect to internet, default qemu bridge device name is virbr0 , due to CentOS doesn’t support brctl, follow this document(3 Ways to Create a Network Bridge in RHEL/CentOS 8) to create a CentOS bridge devices. Below is my experiment script to create bridge device, this setting will keep in system, reboot will not be reset.

nmcli conn add type bridge con-name virbr0 ifname virbr0

killall dhclient
ip addr flush enP7p2s0
nmcli conn modify virbr0 ipv4.addresses '192.168.3.191/16'
nmcli conn modify br0 ipv4.gateway '192.168.1.1'
nmcli conn modify br0 ipv4.dns '8.8.8.8'
nmcli conn modify br0 ipv4.method manual
nmcli conn add type ethernet slave-type bridge con-name bridge-br0 ifname enP7p2s0 master virbr0
nmcli conn up virbr0
nmcli conn up bridge-br0

nmcli conn show  --active

# Allow QEMU to access virbr0
mkdir -p /etc/qemu
echo "allow virbr0" >> /etc/qemu/bridge.conf

Run Qemu

It can refer above to prepare cloud image, I prefer to use ubuntu cloud image, but have not try CentOS image.

Remember Ubuntu 18.04 image won’t auto increase image size, please follow this Change Ubuntu Cloud Image Size to modify image size

qemu-system-x86_64 -m 8192 -smp 16 -nographic \
        -drive if=pflash,format=raw,unit=0,file=OVMF_CODE-pure-efi.fd,readonly=on \
        -drive if=pflash,format=raw,unit=1,file=OVMF_VARS-pure-efi.fd \

        -hdc bionic-server-cloudimg-amd64.img \
        -netdev bridge,br=virbr0,id=net0 \
        -device virtio-net,netdev=net0,mac=00:11:22:33:44:55 \
        -serial telnet::9001,server,nowait > vmlog.txt 2>> vmlog.txt &

x86_64 application

Sometimes we don’t need to run whole VM to run some application just for x86_64, it can run user mode application directly. I am a lazy guy, so just reuse cloud image as x86_64 Linux library.

mkdir -p /home/amd64
qemu-img convert  bionic-server-cloudimg-amd64.img bionic-server-cloudimg-amd64.raw
losetup /dev/loop0 bionic-server-cloudimg-amd64.raw
kpartx -a /dev/loop0
mount /dev/mapper/loop0p1 /home/amd64

Umount Image

umount /home/amd64
losetup -d /dev/loop0

Run x86_64 only application

qemu-x86_64 -L /home/amd64/ bootutil64e
qemu-x86_64 -L /home/amd64/ nvmupdate64e -l

Unsolved Problem

Cannot find way to update INTEL NIC’s NVM

Tried vfio and application mode, vfio cannot pass iommu to host, it will fail.

User space application will cause mmap fail.

If you know how to update Intel NIC’s NVM, welcome to leave a message for me.

QEMU –static problem

Add –static parameter will cause some problem, cannot solve it right now.

./configure –target-list=”x86_64-softmmu aarch64-softmmu aarch64-linux-user x86_64-linux-user” –static
===

ERROR: sizeof(size_t) doesn’t match GLIB_SIZEOF_SIZE_T.

    You probably need to set PKG_CONFIG_LIBDIR

    to point to the right pkg-config files for your    build target

The post QEMU Run x86_64/AMD64 VM on ARM64/AARCH64 appeared first on richliu's blog.

主要還是 yum update 不能用了, 像是

# yum update
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
YumRepo Error: All mirror URLs are not using ftp, http[s] or file.
 Eg. /
removing mirrorlist with no valid mirrors: /var/cache/yum/x86_64/5/base/mirrorlist.txt
Error: Cannot find a valid baseurl for repo: base

這時候就要修改 mirror 檔案, mark 掉所有 mirrorlist 的地方, 改從 valut 抓檔案
編輯 /etc/yum.repos.d/CentOS-Base.repo , mark 掉所有 mirrorlist=
分別加上

baseurl=http://vault.centos.org/5.11/os/$basearch

baseurl=http://vault.centos.org/5.11/updates/$basearch

baseurl=http://vault.centos.org/5.11/extras/$basearch

baseurl=http://vault.centos.org/5.11/centosplus/$basearch

baseurl=http://vault.centos.org/5.11/contrib/$basearch

硬改 5.11 應該也沒有什麼關係，反正也不會再升級或是進版了

好像沒有什麼文章講到這件事, 所以記錄一下

The post Centos 5.X 升級更新問題 appeared first on richliu's blog.

Xorg.0.log
==
(II) Module ddc: vendor=”X.Org Foundation”
compiled for 7.1.1, module version = 1.0.0
ABI class: X.Org Video Driver, version 1.0
(II) VESA(0): VESA VBE DDC supported
(II) VESA(0): VESA VBE DDC Level none
(II) VESA(0): VESA VBE DDC transfer in appr. 0 sec.
(II) VESA(0): VESA VBE DDC read failed
(II) VESA(0): VESA VBE PanelID read successfully
(II) VESA(0): PanelID returned panel resolution 1600×900

Backtrace:
0: X(xf86SigHandler+0x71) [0x4868d1]
1: /lib64/libc.so.6 [0x2b64029a72d0]
2: /usr/lib64/xorg/modules/drivers/vesa_drv.so [0x2b6404b3ef09]
3: X(InitOutput+0x9cd) [0x46431d]
4: X(main+0x26f) [0x4335ef]
5: /lib64/libc.so.6(__libc_start_main+0xf4) [0x2b64029949c4]
6: X(FontFileCompleteXLFD+0x241) [0x432aa9]

Fatal server error:
Caught signal 11. Server aborting
==

Solution:
$ yum install fonts-chinese.noarch

Why, I don’t know.

The post CentOS 5.9 x64 + T420s Xorg crash problem. appeared first on richliu's blog.