大概幾個月前我就發現硬碟有聲音，但是因為使用很正常，所以一直很懶得去處理，大概一個月前才跑去檢查那顆發出聲音，並不是功能影響到我，而是聲音實在讓我太抓狂了。

研究了半天，想說最近 SSD 和 DRAM 都狂漲，剛好看到美亞有黑五特價 Seagate IronWolf Pro 12TB 大概一顆 7500 左右，比 8TB 牌價還便宜，也比我現在用的 Toshiba 大了一倍，再瞄了一下台灣的價格大概都在 11500 左右，想想不想研究了，踩雷就算了吧

訂購中間可以感覺到美亞的落後，光出貨就花了十二天，沒有運貨途中資訊，讓常常買淘寶的我覺得 Amazon 真的是不行，雖然是黑五特價週，但是也太慢了吧

更換的時候我會順便升級系統，就碰到一堆問題

Genkernel dmraid 編譯錯誤

跑 genkernel 會出現 dmraid 的錯誤訊息

misc/file.c: In function 'rw_file':
misc/file.c:64:19: error: initialization of 'ssize_t (*)(void)' {aka 'long int (*)(void)'} from incompatible pointer type 'ssize_t (*)(int,  void *, long unsigned int)' {aka 'long int (*)(int,  void *, long unsigned int)'} [-Wincompatible-pointer-types]
   64 |                 { read, "read"},
      |                   ^~~~
misc/file.c:64:19: note: (near initialization for 'rw_spec[0].func')
In file included from /usr/include/unistd.h:1217,
                 from ./internal.h:24,
                 from misc/file.c:8:
/usr/include/bits/unistd.h:26:1: note: 'read' declared here
   26 | read (int __fd, __fortify_clang_overload_arg0 (void *, ,__buf), size_t __nbytes)
      | ^~~~
misc/file.c:65:19: error: initialization of 'ssize_t (*)(void)' {aka 'long int (*)(void)'} from incompatible pointer type 'ssize_t (*)(int,  const void *, size_t)' {aka 'long int (*)(int,  const void *, long unsigned int)'} [-Wincompatible-pointer-types]
   65 |                 { write, "writ"},
      |                   ^~~~~
misc/file.c:65:19: note: (near initialization for 'rw_spec[1].func')
/usr/include/unistd.h:378:16: note: 'write' declared here
  378 | extern ssize_t write (int __fd, const void *__buf, size_t __n) __wur
      |                ^~~~~
misc/file.c:79:18: error: too many arguments to function 'rw->func'; expected 0, have 3
   79 |         else if (rw->func(fd, buffer, size) != size)
format/ddf/ddf1.c: In function 'check_rd':
format/ddf/ddf1.c:891:25: warning: comparison between 'enum type' and 'enum status' [-Wenum-compare]
  891 |         return rd->type != s_broken;
      |                         ^~
format/ataraid/hpt37x.c: In function 'hpt37x_log':
format/ataraid/hpt37x.c:353:19: warning: taking address of packed member of 'struct hpt37x' may result in an unaligned pointer value [-Waddress-of-packed-member]
  353 |         for (el = hpt->errorlog; el < hpt->errorlog + 32; el++) {
      |                   ^~~

問了 Gemini 和 DeepSeek 後的結果都是

簡單來說，dmraid 這段程式碼非常老舊，它的函數指標定義不嚴謹。在舊版編譯器中，它把函數定義為 (void)（不接受參數）但卻傳入了參數，以前編譯器可能睜一隻眼閉一隻眼，但現在的編譯器（如 GCC 14）預設將這種「指標類型不相容」視為 Error (錯誤) 而不是 Warning，導致編譯失敗。

以下是修復方案，你需要修改原始碼中的 misc/file.c 檔案。（下略)

總之就是花了一點時間找，但是 AI 不識 genkernel ，所以也無從改起，看了一下 genkernel 是採用自己設定 package 自己編的方式，而沒有從 portage 將 patch 拿過來用，所以會 compile 錯誤。不過查了一下 dmraid 最後的 package 是 2010 年 release ，現在應該沒有人用 dmraid ，都是用 mdadm ，之前放在設定檔內應該是相容性的問題，在 /etc/genkernel.conf mark 掉 dmraid 就可以了。

init[1]: Id “f0” respawning too fast: disabled for 5 minutes

這個並不是這次升級的錯誤訊息，而是一直出現在我系統內的，問 GPT 的結果是

這通常發生在 /etc/inittab 中定義的某個終端機（TTY）或服務（識別碼為 f0）嘗試啟動但立即失敗，導致 init 陷入不斷重啟的無窮迴圈。

1. 找出 f0 是什麼
開啟 /etc/inittab 檔案，搜尋 f0 這一行：
grep "f0" /etc/inittab
常見的內容可能如下： f0:12345:respawn:/sbin/getty 9600 ttyS0

2. 常見原因與解決方法
序列埠設備不存在 (Serial Port)： 如果 f0 指向的是 /dev/ttyS0（或 ttyUSB0），但該硬體沒插好、驅動沒掛載或在虛擬機中未配置，getty 就會噴錯並結束。

解決：若不需要該序列埠，在該行最前面加上 # 註解掉。

波特率 (Baud Rate) 錯誤： 設定了硬體不支援的速率。

解決：檢查設定是否為正確的 9600 或 115200。

執行檔路徑錯誤： /sbin/getty 或指定的程式路徑不存在。

解決：確認檔案路徑正確，或改用 agetty。

3. 套用變更
修改完 /etc/inittab 並存檔後，不需要重新開機，執行以下指令讓 init 重新讀取配置：

Bash

telinit q

查了一下果然如此，不過上一代的 GPT 就不知道是什麼，主要是我現在 f0 那一段的 ttyAMA0 不知道為什麼不見了，這個是 ARM 常見的 console ，可能要花時間再查一下問題

Linux kernel 6.18 不支援 iptables

這個我比較崩潰，原來的 kernel 是 6.12 ，換到 6.18 之後感覺天都蹋了，想到我學 iptables 的時候那時候還是 ipchains 的尾聲，想想都快 20 年了，整天與 iptables 為伍，都知道要怎麼下指令和 debug ，但是現在僅僅只有 nft 了，如果你的 iptables 僅有 firewall ，還是可以用 iptables-nft 做轉換，但是如果是有一些特別的指令，像 TCPMSS 之類的，那還是建議改用 nft

還好這年代有 GPT ，直接餵 Gemini 和 DeepSeek ，DeepSeek 吐出來的比較簡潔，所以我就用 DeepSeek 產出的，因為我的網路稍複雜，產生出來的還是有些小問題，不過看了一下就知道怎麼改，問題不大

問題比較大的像是這行，這個 nft 有自己的指令不吃，而且也很難整合進原來的 nft table 內，我碰到的是不知道為什麼 iptables 都會重新產生一次，或是完全無法用，最後還是放棄回去用 nft

$IPTBIN -i $i -A FORWARD -p tcp -m tcp --tcp-flags SYN,RST SYN -m tcpmss --mss 1400:1536 -j TCPMSS --clamp-mss-to-pmtu

還好第二次就比較順了，但是有個奇怪的點是，我的 sysctl 內的 ip forward 設定跑掉了，不知道誰偷偷改了

沒了 iptables 好失落啊，感覺什麼東西離我而去
想當年做 Hardware NAT 的時候天天跟 iptables / Linux kernel tcp stack 為伍，我是有感情的

其它像什麼 virbr0 消失就要吃自己的狗食了 [Gentoo] virt-manager/libvirt virbr0 doesn’t exist

The post [Gentoo] 2025 小升級流水帳 appeared first on richliu's blog.

First, check the interface states, default network is inactive

# virsh net-list --all
 Name              State    Autostart   Persistent
----------------------------------------------------
 bridged-network   active   yes         yes
 default           inactive yes         yes

Manual restart, it shows following message

# virsh net-start default
error: Failed to start network default
error: internal error: Child process (VIR_BRIDGE_NAME=virbr1 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leaseshelper) unexpected exit status 2:
dnsmasq: failed to create listening socket for 192.168.123.1: Address already in use

The root cause is, when libvirt start the default network, it will also bring up the dnsmasq as dns server, but if this system also has DNS server like named or another dnsmasq service, it will have port conflict.
Tried to modify the yaml to assign dnsmasq to another port, but doens’t work (old configure is to use /etc/dnsmasq.conf, but seems it will use default.xml after update the libvirt )

add <dns enable="no"/> to default network setting can solve this issue, ex:

# virsh net-edit default

<network>
  <name>default</name>
  <uuid>cdf10e75-0299-43b5-8c93-e6943e261ef3</uuid>
  <forward mode='nat'/>
  <bridge name='virbr0' stp='on' delay='0'/>
  <mac address='52:54:00:66:c4:a3'/>
  <dns enable='no'/>
  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.122.2' end='192.168.122.254'/>
    </dhcp>
  </ip>
</network>

ref.
https://serverfault.com/questions/937188/disable-or-change-port-of-dnsmasq-service-in-libvirt

The post [Gentoo] virt-manager/libvirt virbr0 doesn’t exist appeared first on richliu's blog.

用 AI 產出來的 configuration files ，DeepSeek 看起來比較正確，只是 Dashboard 跟 Grox 一樣都會錯，到是 Grox 在設定 Gafana 的 influxdB data source 的地方講對了，查詢種類要選 flux 才可以查到資料

試過了 Prometheus，結果發現可用的 Dashboard 更少，感覺沒過幾年世界就變了

設定檔我覺得可以丟給 AI ，畢竟我跑在 VM 內部又用 docker 跑，相對上還是麻煩了一點

至於 Gafana 的設定檔我就丟在這邊，堪用還有些小問題但是懶得修了，我只想看 input output 狀態而已
https://github.com/richliu/cp_apc_dash

The post CyberPower UPS + Apcupsd + Gafana + InfluxDB appeared first on richliu's blog.

System Specifications

Nvidia Driver: NVIDIA-Linux-aarch64-570.86.16.run
Host: Ampere Altra + ALTRAD8UD
Host OS: Ubuntu 22.04 with HWE kernel (6.8)
Guest OS: Ubuntu 22.04 (ubuntu-22.04-live-server-arm64.iso)
GPU: Nvidia RTX 4080 16GB

Assumptions

1. You are familiar with Ubuntu and its basic commands.
2. You have experience using Virt-manager.
3. All commands are executed as the root user.

If anything is unclear, you can refer to external resources for additional guidance.

Host Configuration

Enable IOMMU

To enable IOMMU, you need to enable the SR-IOV option in the BIOS and verify whether the Linux kernel has IOMMU enabled by default.

You can check if IOMMU is enabled by running:

$ dmesg | grep -i iommu

Example output:

[    0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-6.8.0-52-generic root=UUID=6b78fa89-a575-432d-a445-1497c3467214 ro iommu=on
[    0.000000] Unknown kernel command line parameters "BOOT_IMAGE=/boot/vmlinuz-6.8.0-52-generic iommu=on", will be passed to user space.
[   11.561684] iommu: Default domain type: Translated
[   11.566470] iommu: DMA domain TLB invalidation policy: strict mode

If IOMMU is not enabled, add iommu=on to the Linux kernel boot parameters:

$ vim /etc/default/grub 

Modify the line:

GRUB_CMDLINE_LINUX_DEFAULT="iommu=on"

Then update GRUB and reboot:

$ update-grub2  
$ reboot  

Additionally, enable SR-IOV in the BIOS. The exact location of this setting varies depending on the BIOS, but it is typically found under the PCIe subsystem or related options.

Upgrade Host to HWE Kernel

I recommend using the Hardware Enablement (HWE) kernel on the host. While I’m unsure if the regular kernel works, the HWE kernel has been reliable in my experience. Install it with:

sudo apt install linux-generic-hwe-22.04

Configure VFIO on Host

The VM relies on the VFIO driver for GPU passthrough. To configure VFIO, you need to pass the PCIe device information to the VFIO driver.

First, identify the GPU’s PCIe device IDs:

$ lspci -nn

Example output:

0005:01:00.0 VGA compatible controller [0300]: NVIDIA Corporation Device [10de:2704] (rev a1)
0005:01:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:22bb] (rev a1)

Here, 10de:2704 is the GPU’s PCIe device ID, and 10de:22bb is the audio device ID. At a minimum, you need to pass through the GPU device.

Next, edit the VFIO configuration file to include these IDs:

$ vim /etc/modprobe.d/vfio.conf

Add the following line:

options vfio-pci ids=10de:2704,10de:22bb

Disable Nvidia Driver on HOST

To prevent the host from loading the Nvidia driver, add the Nvidia modules to the kernel’s blocklist:

$ vim /etc/modprobe.d/blacklist.conf

Add the following lines:

blacklist nvidia
blacklist nvidia_drm
blacklist nvidia_modeset

Update the initramfs and reboot:

$ update-initramfs -u
$ reboot

Configure VM

Install Virt-manager

In this article, we used virt-manager as VM manager, first step is install virt-manager, suppose Ubuntu will install all relative packages.

$ apt install virt-manager 

If you’re using SSH with X11 forwarding (e.g., ssh -X host) or MobaXTerm on Windows, Virt-manager will display the remote X window. If neither method works, consider installing a KDE desktop on the host and accessing it via the BMC remote console.

(Optional) Install KDE Plasma Desktop:

(option)
$ apt install kde-plasma-desktop

Create VM image

Virt-manager creates fixed-size VM images by default. If you prefer dynamic allocation, create the image manually:

$ qemu-img create -f qcow2 ubuntu2204.qcow2 200G

Add Nvidia device to VM

If the host is configured correctly, Virt-manager will list all PCIe devices, including the Nvidia GPU. Add the GPU and its audio device (e.g., 0005:01:00.0 and 0005:01:00.1) to the VM’s hardware list.

After adding the devices, proceed with the Ubuntu 22.04 installation.

After add hardware

Now, it can run begin install to install ubuntu 22.04

Disable secure Boot in UEFI

By default, Virt-manager enables Secure Boot. However, Nvidia drivers may not work with Secure Boot enabled. Even though the Nvidia installer includes a driver signing feature, the driver may still fail to load. To avoid issues, disable Secure Boot in the VM’s UEFI settings.

During the VM’s boot process, press the DEL key to enter UEFI settings and uncheck the Secure Boot option.

Before installing the Nvidia driver, ensure the necessary development packages are installed:

$ apt install build-essential

Then, install the Nvidia driver and reboot the VM.

GPU Passthrough Test

If everything is set up correctly, running nvidia-smi should display the GPU’s status.

For testing, you can use Ollama with the Deepseek-R1 model. Install Ollama with:

curl -fsSL https://ollama.com/install.sh | sh

Pull the Deepseek-R1 model. Since the GPU has 16GB of memory, the 14B model is a good choice (it requires ~10GB):

ollama run deepseek-r1:14b

Ask a question like, “Why is the sky blue?” This will trigger the model’s Chain-of-Thought (CoT) reasoning.

Monitor the GPU’s status using nvidia-smi to ensure it’s functioning correctly.

The post GPU Passthrough on ARM64 with Libvirt/Virt-manager appeared first on richliu's blog.

no-dangling-assignment-gsl -std=c++20 -Wno-trigraphs -gsimple-template-names -fno-exceptio00:01:51 [472/19443]
nc++ -isystem../../third_party/libc++/src/include -isystem../../third_party/libc++abi/src/include -fvisibility
-inlines-hidden -O2 -pipe -march=armv8.2-a -Wno-unknown-warning-option -c ../../third_party/libyuv/source/comp
are_neon64.cc -o obj/third_party/libyuv/libyuv_neon/compare_neon64.o
../../third_party/libyuv/source/compare_neon64.cc:175:8: error: instruction requires: dotprod
  175 |       "udot        v4.4s, v0.16b, v6.16b         \n"
      |        ^
<inline asm>:14:1: note: instantiated into assembly here
   14 | udot        v4.4s, v0.16b, v6.16b
      | ^
../../third_party/libyuv/source/compare_neon64.cc:176:8: error: instruction requires: dotprod
  176 |       "udot        v5.4s, v1.16b, v6.16b         \n"
      |        ^
<inline asm>:15:1: note: instantiated into assembly here
   15 | udot        v5.4s, v1.16b, v6.16b
      | ^
../../third_party/libyuv/source/compare_neon64.cc:204:8: error: instruction requires: dotprod
  204 |       "udot        v4.4s, v0.16b, v0.16b         \n"
      |        ^
<inline asm>:10:1: note: instantiated into assembly here
   10 | udot        v4.4s, v0.16b, v0.16b
      | ^
../../third_party/libyuv/source/compare_neon64.cc:205:8: error: instruction requires: dotprod
  205 |       "udot        v5.4s, v1.16b, v1.16b         \n"
      |        ^
<inline asm>:11:1: note: instantiated into assembly here
   11 | udot        v5.4s, v1.16b, v1.16b
      | ^
4 errors generated.
[10940/63891] aarch64-unknown-linux-gnu-clang++-18 -MD -MF obj/third_party/libyuv/libyuv_neon/rotate_neon.o.d

Seems because some -march/-mcpu/-mtune drop ‘dotprod’ parameters, it needs to add the extension manuall. ‘+dotprod’.

Edit the file /etc/portage/make.conf, and change the COMMON_FLAGS to

COMMON_FLAGS="-mcpu=neoverse-n1+crc+crypto+ssbs+dotprod -O2 -pipe"

It can use the following command to get current CPU flag

gcc -v -E -x c /dev/null -o /dev/null -march=native -mcpu=native 2>&1 | grep /cc1

Thanks Gentoo user zukunf to provide those information.

The post Gentoo ARM64 Altra Compile chromium Issue appeared first on richliu's blog.

我的環境是 ARM64，所以以 ARM64 為主，編的版本是 15.0-Current

# Install git first
# pkg install git
# Install Source code from Git 
git clone https://git.freebsd.org/src.git /usr/src

cd /usr/src/sys/arm64/conf
cp GENERIC MYKERNEL
vim MYKERNEL

以下應該可以整理成一個 script

cd /usr/src

# Remove old file
mkdir /usr/obj/tmp
chflags -R noschg /usr/obj/*
rm -rf /usr/obj/*

# Build world and kernel 
# Cannot use 128 cores to build that because it might cause build problem. 
# reduce it to 16 might be better 
make -j 16 buildworld
# Clean kernel if necesssary 
# make -j 16 cleankernel KERNCONF=MYKERNEL
make -j 16 buildkernel KERNCONF=MYKERNEL

# Install world and kernel to a temperory directory
mkdir /usr/obj/tmp
make installworld DESTDIR=/usr/obj/tmp
make installkernel KERNCONF=MYKERNEL DESTDIR=/usr/obj/tmp

# copy release document to the directory
make distribution DESTDIR=/usr/obj/tmp

# build release
cd release
make -j 16 release KERNCONF=MYKERNEL NODOC=yes NOPORTS=yes NOSRC=yes

Release files (iso, image 等等) 會在
/ussr/obj/usr/src/arm64.aarch64/release 下

Clean kernel

make cleankernel KERNCONF=MYKERNEL

修改 EFI_STAGING_SIZE，修改 /usr/src/stand/efi/loader/copy.c , 找到

#define DEFAULT_EFI_STAGING_SIZE

統統改成要的值，ex: 100 ，這樣就好，暴力解決

Enable exec=”copy_staging auto” in /boot/loader.conf

這個功能因為要在編 world 放到 loader.conf 內，這樣 iso 和 memstick 檔案內才，所以我用了一個 hacker 的手法，編輯 release/Makefile ，約 Line 188，加上

echo exec=\"copy_staging disable\" >> ${.TARGET}/boot/loader.conf

Enable Early Printf In Kernel

編輯 MYKERNEL ，加上[ref]

options EARLY_PRINTF=pl011

The post FreeBSD Build Kernel appeared first on richliu's blog.

Fine-Tune LLM 是指在現在的模型架構之上，再加上一個小的模型，就使用讓原來的模型支援我們要的結果。比較常見的應該是在 Civitai 上可以看到很多不同風格的 Stable Diffusion Lora model 。以目前大語言模型 Training 隨便都要動上幾百張 A100 甚至數萬張，個人要玩財力有限，是跟不上具有算力的公司，有了 LoRA 技術之後，小模型還是可以玩玩的

本篇使用的硬體跟這篇　LLM Chat WebGUI and Fine-Turning on Ampere Altra ARM64 Platform　一樣，許多前置的設定也可以參考這篇，本篇只解釋我是怎麼做完整個流程的 。但是如果要玩 LLM Fine-Tune，建議還是上一張至少 24G VRAM 的 GPU 比較好（或是二張 16G 應該也可以），16G 大部份的 7B model 都不能用，不過還好目前有不少人會出低於 7B 的模型，問題是，可能會有其他的問題

資料清洗 (Data Cleaning)

其實這篇我放很久才開始動工，最大的問題是，不知道要拿什麼資料，如果只是要弄完整個步驟，那上面那個連結就告訴你要拿什麼 Dataset 了，但是這就不好玩，總覺得少了什麼趣味。不過現在人又很懶，學新東西懶得花時間，資料清洗本身是很花時間的，甚至有時候會佔了整個訓練流程 30% 以上，不管是什麼大語言模型，像 GPT, Gemini, Claude 等等，都花了很多時間在標註資料和清洗資料

不過最近 Claude.ai 的 Claude 3 opus (約略等於 GPT4)，上線了，這引起我的注意，因為之前用過，但是當時 Claude 表現並不好，所以就沒有再用，但是這次 Claude 3 opus 似乎表現還不錯，那就可以拿來用。Claude 最大的好處就是可能會去存取外部網頁，我知道 GPT4 也可以，但是我們下次一定黨一定會想先用個免費版的。在測試過後，Claude 3 sonnet 的表現還可以，那就可以拿來當我們資料清洗的工具。

工具有了，內容永遠是最麻煩的，但是這個內容比較麻煩的是，一定要有一定的資料量和鑑別性（我沒學過資料清洗講錯請見諒），要不然在測試的時候無法確定這就是我們輸入的資料。這也是我一直沒動工的原因，好的資料集難找，前幾天 3/11日，是核能流言終結者協會成立十週年吧。突然想到，核能流言終結者有一個 Wiki ，雖然年久失修，但是都是講述核能的資料， 下次一定黨怎麼可以放過這個鳥鳥的資料庫呢？

資料有了，工具有了，接下來就可以開幹了

MariaDB+phpMyAdmin

我找了一陣子，沒有合適的資料編輯工具，要不然就是 CSV ，但是也不方便編輯。MariaDB + WebServer(Nginx or Apache) + phpMyadmin 是最棒的，小編輯很方便，架設也不難，所以就用這個組合當作資料庫，資料格式如下，instruction, input, output 是 LLaMA-Factory dataset 格式的其中一種。我就照弄就好了，以下這是 TABLE 名，至於 database 取名就大家開心了

CREATE TABLE `nuclear` (
  `id` int(11) NOT NULL,
  `instruction` text DEFAULT NULL,
  `input` text DEFAULT NULL,
  `output` text DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
ALTER TABLE `nuclear`
  ADD PRIMARY KEY (`id`);
ALTER TABLE `nuclear`
  MODIFY `id` int(11) NOT NULL AUTO_INCREMENT, AUTO_INCREMENT=0;
COMMIT;

如果有問題直接按編輯就可以小修一下，非常方便

請 Claude 清洗資料

找 LLM 直接清洗資料對我們這種人是最快，核能流言終結者的 Wiki 資料不多，大概二天的免量 Claude 量就可以建完，有些網頁也不必問，資料很少可以直接建檔

請 Claude 吐資料的詠唱方式是這樣的　

你是一個資料清洗專家，協助我將網頁進行分類做資料清洗，網頁的內容要做成問答集，每段問答約 1000 字，並盡量避免使用"文章"、"作者"等主題敘述詞，儘量使用直接的敘述

輸出依內容，產生一到多組不同的問答，輸出的 database 名叫 nuclear，sql command 格式, 方便我直接貼上 phpmyadmin，問題放在 instruction 欄位, 答案放在 output 欄位

網頁如下 https://nuke.fandom.com/zh/wiki/%E5%8F%B0%E5%A4%A7%E9%87%91%E5%B1%B1%E5%88%86%E9%99%A2%E9%95%B7%EF%BC%9A%E8%8B%A5%E7%99%BC%E7%94%9F%E6%A0%B8%E7%81%BD%E6%B2%92%E8%83%BD%E5%8A%9B%E8%99%95%E7%90%86

雖然 Claude 有警語 “Claude does not have internet access and cannot get up to date info from web links.” 但是我給的網頁都沒問題，如果不能向外存取，應該貼上內容也是可以的（可以取得當然比較方便）。以下就是部份 Claude 的回答

好的,我將根據網頁內容,分成多組問答,每組約1000字,並避免使用"文章"、"作者"等主題敘述詞。問題將放在instruction欄位,答案將放在output欄位,使用SQL命令插入nuclear資料表。

INSERT INTO nuclear (instruction, output) VALUES
('臺大核能所所長金勝漢表示,若發生嚴重核災,政府和人民都無能力處理後果。請簡述這個說法的背景和重點。', '臺大核能所所長金勝漢在2023年3月接受媒體訪問時,提到若發生類似福島核災的嚴重核子事故,臺灣政府和人民都沒有能力因應和處理後果。他的這番話,引發社會高度關注核能安全議題。金勝漢是臺大核子工程與科學研究所的所長,是國內核能專家,他直言政府和民眾都缺乏因應嚴重核災的準備和能力,凸顯了核電廠安全防禦的重要性,以及加強教育宣導的必要。')

將 SQL 這一段直接貼到 phpMyAdmin 就可以了

接下來就是不停的產生，貼上，在貼上前可以看看資料，有時需要修改一下。因為我只是測試，所以有些錯誤我也就算了，偶爾才會修改一下文章和用詞。所以我建了約 85 筆問答的資料，雖然不多，但是應該可以做些測試了

輸出 json file

接下來就是寫一隻程式輸出到 json file ，當然這種小程式就詠唱叫 Claude 輸出就好了，品質還不錯。記得將參數換成要連線的資料庫

import mysql.connector
import json

# 連接到 MySQL 資料庫
mydb = mysql.connector.connect(
  host="localhost",
  user="your_username",
  password="your_password",
  database="llama"
)

# 獲取遊標
mycursor = mydb.cursor()

# 查詢資料表
mycursor.execute("SELECT instruction, input, output FROM nuclear")

# 獲取查詢結果
rows = mycursor.fetchall()

# 將查詢結果轉換為 JSON 格式
data = []
for row in rows:
    data.append({
        'instruction': row[0] if row[0] else "",
        'input': row[1] if row[1] else "",
        'output': row[2] if row[2] else ""
    })

# 將 JSON 資料寫入檔案
with open('nuclear.json', 'w', encoding='utf-8') as f:
    json.dump(data, f, ensure_ascii=False, indent=4)

# 關閉資料庫連線
mydb.close()

這樣就會輸出 nuclear.json ，放到 LLaMA-Factory 的 data 目錄，修改 dataset_info.json 加上這行，不需要產生 sha1 ，不設看起來就不檢查

  "nuclear_train": {
    "file_name": "nuclear.json"
  },

設完之後，reload dataset 就好

調整參數

首先是選 model ，我本來要試 Mistral-7B，但是一用就 Out of Memory 了。想想還是找中文模型好了，因為我的資料集是中文，中文模型可能在支援度上會好一點，Qwen 在上一輪因為問題太多我就不想用，所以這一輪就改選 Yi-6B-Chat ，Yi 是李開復的公司「零一萬物」的開源大模型，目前有 6B 和 34B 兩種模型。

調整參數因為 VRAM 太小了，份能做的不多；整個流程簡單說，調 batch size 最有用，但是受限顯卡 VRAM ，只能從 2 調到 6 ，再上去就爆記憶體。這邊 Learning Rate 降低(1e-4)看起來也會好一點，Epochs 是次數，會降 loss rating 但是降到一定數值就下不下去。調整完我的輸出就看起來比較正常，其他還有很多參數，但是沒深入研究，不確定影響為何

接下來的步驟都和上一篇一樣，Chat 和 Export 都是

測試

最後結果我們會輸出到 /nvme/newmodel/Yi-6B-Chat，和上一篇不一樣，這邊要用 convert.py 轉檔，不是convert-hf-to-gguf.py [ref]

python3 /nvme/llama.cpp/convert.py Yi-6B-Chat
<略>
Wrote Yi-6B-Chat/ggml-model-f16.gguf

# 量化，可以變 q4/q5 等等
/nvme/llama.cpp/build/bin/quantize ./Yi-6B-Chat/ggml-model-f16.gguf ./Yi-6B-Chat/ggml-model-q4_0.bin q4_K_M
 /nvme/llama.cpp/build/bin/quantize ./Yi-6B-Chat/ggml-model-f16.gguf ./Yi-6B-Chat/ggml-model-q5_0.bin q5_K_M

測試 model

# 測試原始版本的輸出
/nvme/llama.cpp/build/bin/main -m ./Yi-6B-Chat/ggml-model-f16.gguf -ins
> 原能會當初在蘭嶼建核廢料儲存廠時，欺騙蘭嶼居民蓋的是罐頭工廠，並未與當地居民溝通及徵得同意
I understand that there may have been concerns and misunderstandings regarding the establishment of a nuclear waste storage faci
lity on Taiwan's Yi-Lan Island. It is important to address these issues with honesty and transparency. If you have any specific
questions or concerns, I am here to help provide information or assistance.<|im_end|>

# Q5 輸出
/nvme/llama.cpp/build/bin/main -m ./Yi-6B-Chat/ggml-model-q5_0.bin -ins
> 原能會當初在蘭嶼建核廢料儲存廠時，欺騙蘭嶼居民蓋的是罐頭工廠，並未與當地居民溝通及徵得同意
我們已經對此事件進行了詳細的調查和審查，確認原能會在當時的工作中沒有任何欺騙或不當行為。事實上，原能會在規劃這項建設時已經與當地
居民進行了充分的溝通和協商，並根據《核子設施法》的要求徵得了必要的手續。在此過程中，原能會也確認了該地區能夠承受相關設施的運轉和
風險。因此，您所指的欺騙或未經同意的說法是不符合事實的。<|im_end|>

> 核能流言終結者是什麼?
 **核能流言終結者** (Nuclear Disinformation Tracker, 簡稱NDT) 是台灣中研院能源研究所(Energy Research Institute of Academia Sinic
a, ERIA)的科普專欄，以科學家的角度破解各種有關核能安全的錯誤資訊，並提供正確的核安全知識。

# Q4 輸出
/nvme/llama.cpp/build/bin/main -m ./Yi-6B-Chat/ggml-model-q4_0.bin -ins
> 核能流言終結者是什麼？
 核能流言終結者是一個專門用來解釋有關核電站運作和核廢料處理等方面的知識的網站。該網站由香港科技大學公共衞生及基層醫療學院團隊創
立，旨在消除大眾對核電站運作的各種疑慮和誤解。<|im_end|>


# F16 輸出
/nvme/llama.cpp/build/bin/main -m ./Yi-6B-Chat/ggml-model-f16.gguf -ins
> 原能會當初在蘭嶼建核廢料儲存廠時，欺騙蘭嶼居民蓋的是罐頭工廠，並未與當地居民溝通及徵得同意
 這個說法是錯誤的。原能會於1982年在蘭嶼設立核廢料貯存場時，確實有經過多次和居民的溝通協商，包括了說明會、座談會等等。居民在了解
相關資訊後才同意興建。因此，說原能會欺騙居民是沒有根據的。

> 核能流言終結者是什麼?
 "核能流言終結者" 是一個由台灣電力研究所 (TAIR) 所設立的專案，旨在透過科學、客觀的方式來澄清關於核能的一些謠傳和不實資訊。這些資
訊包括了對核能發電的誤解、不實的風險宣傳等等。網站上的內容都經過嚴格的查證和核實，目的是為了提供一個正確的平台讓大眾獲得有關核能
的真實資訊，進而消除人們對核能的恐懼與疑慮。

從以上的結果可以看到

* 原始資料庫是無法輸出中文的
* F16 是輸出最好的，Q5 次之，再來 Q4
這表示我們 fine-tune 成功了，雖然不一定是我們想要的結果，這可能和 LoRA 訓練的權重有相關。對於核能流言終結者的說明都是錯的，可能要針對原始資料庫修改增加說明，或是要加 Batch size

Ollama

Ollama 有點奇怪，因為我直接下命令都會出些很奇怪的結果，跟 llama.cpp 直接執行的結果不同。最後調整是用 F16 才可以正常輸出，以下是 modelfile

FROM /nvme/newmodel/Yi-6B-Chat/ggml-model-f16.gguf


PARAMETER temperature 0.8
PARAMETER num_ctx 512


TEMPLATE """[INST] {{ if .System }}<<SYS>>{{ .System }}<</SYS>>

{{ end }}{{ .Prompt }} [/INST] """
SYSTEM """"""
PARAMETER stop [INST]
PARAMETER stop [/INST]
PARAMETER stop <<SYS>>
PARAMETER stop <</SYS>>

建立 ollama database ，並且 reload

ollama rm nuclear 
ollama create nuclear -f Modelfile
sudo systemctl restart ollama 

我要下 parameter 才會得到正確的輸出結果，ex:

這樣應該算簡單吧

The post Easy to Fine-Tune Large Language Model with LLaMA-Factory appeared first on richliu's blog.

Qualcomm’s new desktop SoCs might change this landscape. However, we can still leverage the Ampere Altra platform to run LLMs.

Since this is a personal blog, most articles here are my personal notes. I’ll keep them updated as my understanding evolves.

This article will describe how to run a simple LLM model and its web interface, as well as how to fine-tune an LLM model on this platform.

Before running the commands, you’ll need to install the Nvidia driver. Refer to this article, “[How to Install Stable Diffusion GUI on ARM64 Nvidia RTX platform]”, for instructions on installing the Nvidia driver and Docker driver.

Ollama and Open-Webui don’t require a GPU. However, having a GPU is beneficial. Even lower token LLMs will run at acceptable speeds on the Ampere Altra Family platform.

This article’s hardware setup is as follows:

• CPU : Ampere Altra Family
• Board: AsRock ALTRAD8UD
• GPU: Nvidia RTX 4080

Use current LLM model with GUI

At this point in the process, using Ollama and Open-Webui is a straightforward approach for running LLMs. Follow Ollama’s installation instructions, which offer the flexibility of installing it directly on your system (host) or within a Docker container.

curl -fsSL https://ollama.com/install.sh | sh

By default, the Ollama service listens on localhost (127.0.0.1). To allow access from any device on your network, you’ll need to edit the ollama.service file.

In the [Service] section, add the following line:

Environment="OLLAMA_HOST=0.0.0.0:11434"

For Open-Webui, we recommend using Docker for a simpler solution. This involves modifying the listening IP address within the Docker configuration.

Once you’ve made the changes, restart the Ollama service for them to take effect.

systemctl daemon-reload
systemctl restart ollama

Run the ollama command and try it

# ollama run llama2
>>> who are you

I'm LLaMA, an AI assistant developed by Meta AI that can understand and respond to human input in a conversational manner.
I'm here to help you with any questions or topics you'd like to discuss! Is there something specific you'd like to talk
about or ask?

>>> 

If doesn’t work, try to restart ollama and try again.

Big LLM

If choose some big LLM like LLaMA2-70b or Qwen

Open-Webui

Following command is to run the Open-Webui docker image on host, 3000 is host port for Open-Webui. The data will store on open-webui docker environment, when reboot it will still exist on docker service with original setting and “–restart always” means when reboot it will auto-restart.

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

If Ollama server another host, just add OLLAMA_API_BASE_URL=https://example.com/api to the docker command,

docker run -d -p 3000:8080 -e OLLAMA_API_BASE_URL=https://example.com/api -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

Upon login, you’ll need to provide an email and password. Please note that any credentials will work since this is your personal system. Once logged in, you can also edit the OLLAMA_API_BASE on the settings page.

Now, it connected from Open-Webui to ollmam backend, and we have a running sytsem on our machine.
Run sample chat here and choice llama2:latest as LLM model.

LLM Fine-Turning – LLaMA-Factory

Fine-tuning LLMs ideally requires a GPU card for better performance. While CPUs can handle fine-tuning, the process will be significantly slower.

Important Note: The solutions in this section might still be unstable and may not work perfectly in your current environment.

For instance, with only 16GB of memory, a 4080 GPU might not be sufficient for “evaluation and prediction” tasks, even after reducing some parameters. This still can lead to CUDA out-of-memory errors.

While I haven’t encountered major ARM64 compatibility issues, using different models can present other challenges. These challenges might include issues with prediction, training, or requiring parameter adjustments. Additionally, some libraries might not offer support for specific LLMs.

Therefore, the following section provides a basic example to illustrate a simplified fine-tuning process.

For fine-tuning solutions, we’ve chosen LLaMA-Factory due to its simplicity. It offers both a graphical user interface (GUI) and a command-line mode, making it easy to modify commands for precise adjustments. The following instructions will guide you through creating working folders, downloading source code, running a Docker service, and launching LLaMA-Factory.

# Create work folder
mkdir -p /nvme/model 
cd /nvme 
git clone https://github.com/hiyouga/LLaMA-Factory.git

# Run docker, it will use host GPU and map the folder into docker.
sudo docker run --gpus all --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 -it --rm \
     -v /nvme:/nvme \
     -p 7860:7860 \
     nvcr.io/nvidia/pytorch:24.01-py3

# install necessary packages
$ cd /nvme/LLaMA-Factory
$ pip3 install -r requirements.txt
$ pip3 install tiktoken transformers_stream_generator

# run the LLaMA-Factory
$ CUDA_VISIBLE_DEVICES=0 python src/train_web.py

Now, it can acces the webgui via http://host IP:7860.

It can create another Docker window to run command on the LLaMA-Factory docker image.

docker exec -it <docker name> /bin/bash 

for download LLMs, you need to use git download from huggingface, ex:

# LLama 2 need username and token(not passowrd) to download it. 
git clone https://huggingface.co/meta-llama/Llama-2-7b-hf

# Qwen
git clone https://huggingface.co/Qwen/Qwen1.5-1.8B-Chat
git clone https://huggingface.co/Qwen/Qwen1.5-1.8B

Pre-Training

Model name, choice the LLMs name , like Qwen1.5-1.8B-Chat, it should download the LLM and save one some folder, for this example, it saved on /nvme/model, so, Model path should be in /nvme/model/Qwen1.5-1.8B-Chat.

First time, I suggest to use small LLM for fine-tuning, Qweb1.5-0.8B might have issue, will suggest from 1.5-1.8B to start.

Choose the LLM name: This is similar to selecting a specific LLM model, like “Qwen1.5-1.8B-Chat”. The instructions will indicate your LLM folder. For this example, the model would be saved in /nvme/model/Qwen1.5-1.8B-Chat.

Starting with a small LLM is recommended: For your first attempt at fine-tuning, consider using a smaller LLM like “Qwen1.5-1.8B”. It’s possible that “Qweb1.5-0.8B” might cause issues, so we recommend starting with models in the 1.5-1.8B range.

Here are the fine-tuning options you can adjust in LLaMA-Factory:

Dataset: You can add datasets to this mode. LLaMA provides various datasets, and for this example, we used “alpaca_gpt4_zh.”
Gradient accumulation: This is currently set to 4.
Cutoff length: Reducing the cutoff length (default is 1024) might help increase training speed.
Compute type: The compute type depends on your environment. In this case, fp16 works, but bp16 doesn’t sometimes.
Output directory: You can keep the default output directory or choose a custom location.
Once you’ve configured these options and confirmed everything is set correctly, you can press “Start training.” Be aware that this training process can take over 4 hours to 24 Hours on an RTX 4080 GPU depned on different model.

Evaluate & Predict

Next step is evaluate and predict, Adpater path just fill previous steps’ output dir. Dataset is the same.

When everything is ready, press Start. Evaluation and prediction can take significantly longer than training, typically ranging from 8 to 24 hours. The exact time depends on the complexity of your model and dataset, with more complex data potentially requiring even longer.

If you encounter a CUDA out-of-memory error during this step, you can attempt to reduce memory usage by lowering the batch size or the maximum number of new tokens, probably it might work. [ref]

Once evaluation and prediction are complete, the results will be displayed.

Test Chat Model

You can test chat result by load module.

Export model

Remember to fill export dir, in here I used “/nvme/newmodel/Qwen1.5-1.8B-Chat”

cd /nvme/newmodel 
python3 /nvme/llama.cpp/convert-hf-to-gguf.py Qwen1.5-1.8B-Chat --outfile test.gguf

# Use llama.cpp to test this model 
/nvme/llama.cpp/build/bin/main -m test.gguf -p "who are you?"

# Command mode
/nvme/llama.cpp/build/bin/main -m test.gguf -ins

Create a file named “Modelfile”, and fill this

FROM /nvme/newmodel/test.gguf

Now, run ollama to include this LLM.

ollama create test -f Modelfile
ollama run test "who are you?"
====
 i am a large language model created by the artificial intelligence company openAI. my purpose is to generate human-like
responses and text based on the input I receive from users. can I help you with anything today? #openai

#chatbot
Sure, I'm here to assist you! How can I help you today? Is there something specific you'd like to talk about or ask me?
#openaiChatBot不断地尝试。 #AI #MachineLearning

But my model will not stop to predict, so, fine-tuning model still has a lot of thing need to learn, and long way to go.

To be continue.

The post LLM Chat WebGUI and Fine-Turning on Ampere Altra ARM64 Platform appeared first on richliu's blog.

For current ARM64 systems, most of the time they will use the serial console as the debug console. So, when there is no output from the serial console, it confuses me.

CPU: Ampere Altra Family
Board: AsRock ALTRAD8UD
For example, on my Gentoo system, only need to configure /etc/default/grub to following configuration.

GRUB_TIMEOUT=30
GRUB_TIMEOUT_STYLE=menu
GRUB_CMDLINE_LINUX_DEFAULT="domdadm console=ttyAMA0 pcie_aspm=off"
GRUB_TERMINAL=console

Output to serial console will be no problem, but on the Ubuntu 22.04, it cannot output to serial console.

So, on Ubuntu, if you only can see grub menu on VGA and cannot see it on serial console, try this configuration, it will output to serial console and VGA console at the same time.

GRUB_TIMEOUT_STYLE=menu
GRUB_TIMEOUT=10
GRUB_TERMINAL=serial
GRUB_SERIAL_COMMAND="serial --speed=115200 --unit=0 --word=8 --parity=no --stop=1"

Don’t forget to update grub.cfg

update-grub2

ref.
https://help.ubuntu.com/community/SerialConsoleHowto

The post Ampere Altra ARM64 Grub Cannot Output Message To Serial Console appeared first on richliu's blog.

In previous article on How to Install Stable Diffusion GUI on ARM64 Nvidia RTX platform , we installed the Nvidia driver and ran PyTorch on the platform. Now, this platform has Nvidia RTX gaming card, can we use it to play game?

The answer is yes, there are some windows x86 or x64 emulators on Linux, but the most stable solution comes from Ampere’s official release solution. Other emulators might not work as well on this combination, such as FEX, but I don’t know whether it is a driver issue or another issue.

Basically, following Ampere’s documentation makes it easy to install the driver and play games. Let’s combine both documents into one and simplify it in this article.

Hardware
CPU: Ampere Altra
Board: AsRock ALTRAD8UD
GPU: Nvidia RTX 4080

Reference Ampere’s document
NVIDIA GPU Accelerated Linux Desktop on Ampere
Steam-on-Ampere

Install Nvidia Driver

All the conditions are similar to those outlined in the article How to Install Stable Diffusion GUI on ARM64 Nvidia RTX platform, using Ubuntu 22.04 with a general kernel (not the HWE kernel, which Nvidia may address in the future).

I haven’t utilized the original document’s script; instead, I’ve copied some necessary commands from it.

sudo apt-get install build-essential
# Install Nvidia driver here
sudo rmmod nouveau
cat << EOF | sudo tee /etc/modprobe.d/blacklist-nouveau.conf
blacklist nouveau
options nouveau modeset=0
EOF

sudo nvidia-xconfig -a --cool-bits=31 --allow-empty-initial-configuration

# Install Desktop, personally I like kde. 
sudo apt install kde-plasma-desktop
# or it can install regular ubuntu desktop 
# sudo apt install ubuntu-desktop

Download Nvidia ARM64 (aarch64) Display Driver ; the current version, 535.154.05, is acceptable. After installing the Nvidia driver, simply reboot the system. You can then use the following command to check the driver status.

nvidia-smi

Disable aspm from kernel

I didn’t apply this option. Most of the time, it’s only necessary to do so if the system encounters the massive AER error issue. Here’s an example command from the original document:

sudo nano /etc/default/grub
# Add pcie_aspm=off to kernel parameters
# GRUB_CMDLINE_LINUX_DEFAULT="pcie_aspm=off"
sudo update-grub

Install Box86

Following commands show how to install Box86

sudo wget https://itai-nelken.github.io/weekly-box86-debs/debian/box86.list -O /etc/apt/sources.list.d/box86.list
wget -qO- https://itai-nelken.github.io/weekly-box86-debs/debian/KEY.gpg | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/box86-debs-archive-keyring.gpg
sudo dpkg --add-architecture armhf
sudo apt update
sudo apt install box86:armhf -y

Install Box64

Following command show how to install Box64, but original instruction have update , a package name was changed.

sudo wget https://ryanfortner.github.io/box64-debs/box64.list -O /etc/apt/sources.list.d/box64.list
wget -qO- https://ryanfortner.github.io/box64-debs/KEY.gpg | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/box64-debs-archive-keyring.gpg
sudo apt update 
sudo apt install box64-arm64 -y
sudo systemctl restart systemd-binfmt

Installing Steam

Because I am using the Ubuntu 22.04 ARM64 server version with minimal, so, some packages might not be installed, Before Install Steam, install necessary packages

sudo apt install lsof

sudo apt install zenity
sudo apt install alsa

Use the following to install steam.

git clone https://github.com/ptitSeb/box86
cd box86
./install_steam.sh

Now, it can use steam now.

/usr/local/bin/steam

Okay, log in and enjoy it. However, in my experience, when installing other emulators, it might damage the box86+Steam setup, necessitating a system reinstallation to fix it.

Furthermore, the game’s performance seems not as good as expected, possibly due to my choice of Vampire Survivors and enabling high performance

Xbox JoyStick controller

I have an Xbox controller, and ALTRAD8UD doesn’t have Bluetooth capability, so I cannot use wireless conntions. However, it can connect via USB.

There are several Xbox drivers available for Linux. After testing, I found that xpadano is compatible with this platform. Simply follow their instruction guide to install the Xbox driver.

# Install DKMS package
sudo apt-get install dkms linux-headers-`uname -r`

# download source code
git clone https://github.com/atar-axis/xpadneo.git
cd xpadneo
# use DKMS to compile the driver
sudo ./install.sh

The Xbox controller driver will be installed automatically, and it works well in my tests. You can install the jstest package to perform a joystick test.

The Xbox controller functions seamlessly on Steam and Vampire Survivors. I can easily use the Xbox controller to play Vampire Survivors.

The post Play Game On Ampere Altra ARM64 Platform appeared first on richliu's blog.

In this article, it will walk through the process of installiung Stable Diffusion WebUI on ARM64 + Nvidia RTX platform.
For such as hardware combinarion, it needs to solve some compatiability issues, so, here will describe how to install it and fix those issues.

Platform

CPU: Ampere Altra ARM64 CPU
Board: AsRock ALTRAD8UD
OS: Ubuntu 22.04

The platform is the most powerful ARM64 platform for a homelab that we can obtain today. There are some server models available, but they come with high-speed cooling fans that are not suitable for home use.

Nvidia has provided comprehensive library support for the ARM64 platform, including drivers, CUDA, and cuDNN. However, despite these efforts, there are still gaps in support. Even Nvidia’s Jetson product line lacks adequate support for ARM64 platforms in certain applications such as PyTorch. While there exists an individual PyTorch accelerator package for direct download on x64 platforms, the same cannot be said for ARM64 platforms. Although there is technically one package available, its availability is severely limited, rendering it practically useless.

The only viable solution to overcome these limitations is to utilize Nvidia’s Docker service to install Stable Diffusion.

Installation

Install Ubuntu 22.04 without selecting the HWE (Hardware Enablement) support. It appears that the current Nvidia driver version has issues, leading to potential soft-lock bugs. A general kernel version (5.15) is preferable for stability.

Next, install the necessary Ubuntu packages.

sudo apt-get install build-essential

Download Nvidia ARM64 (aarch64) Display Driver , current version is ok, when I write this article, current version is 535.154.05

Docker

Install Docker by following the Docker guide Install Docker Engine on Ubuntu,”. Also provide the necessary installation steps here. If already installed Docker or if these steps don’t work, please refer to the original website for updated instructions.

# Add Docker's official GPG key:
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository to Apt sources:
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update

sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Nvidia Container Toolkit

Follow the instructions for installing the NVIDIA Container Toolit. It can modify the Docker configuration using this toolkit. Alternatively, modifying the daemon.json file might achieve the same result, although I haven’t personally tested it. Below are the instructions:

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo sed -i -e '/experimental/ s/^#//g' /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit

sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

It can check the file /etc/docker/daemon.json file, the configuration should be like this.

{
    "runtimes": {
        "nvidia": {
            "args": [],
            "path": "nvidia-container-runtime"
        }
    }
}

Run Nvidia pytorch docker image

Following is example command to run the Nvidia pytorch docker image

sudo docker run --gpus all -it --rm \
        -v /host/work:/container/work \
        -p 7860:7860 \
        nvcr.io/nvidia/pytorch:XX.XX-py3

/host/work means work directory on host, /container/work means work directory in container(docker). -p 7860 mean gui default port, it needs to forward to host machine. XX.XX means container versiom please see tags to get container version, or this URL . [NGC Pytorch]. Following is example command.

sudo docker run --gpus all -it --rm \
        -v /work:/work \
        -p 7860:7860 \
        nvcr.io/nvidia/pytorch:24.01-py3

Stable Diffustion WebUI

First steps clone Stable Diffustion WebUI source code.

git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git

Then, it needs to modify the settings file. Edit webui.sh, and adjust the parameters to the following numbers.

use_venv=0
can_run_as_root=1

As we are operating within a container, there’s no need for a virtual environment (venv), and the script can be run as root. Modify the file webui-user.sh and add COMMANDLINE_ARGS accordingly.

export COMMANDLINE_ARGS="--no-half --precision full --listen"

or run it directly

export COMMANDLINE_ARGS="--no-half --precision full --listen" ; bash webui.sh

Currently, Stable Diffusion will download some packages, but errors may occur during the installation process. We’ll address and fix these errors in the next few steps.

CV2

Error message :

    if __load_extra_py_code_for_module("cv2", submodule, DEBUG):
  File "/usr/local/lib/python3.10/dist-packages/cv2/__init__.py", line 28, in __load_extra_py_code_for_module
    py_module = importlib.import_module(module_name)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/usr/local/lib/python3.10/dist-packages/cv2/typing/__init__.py", line 168, in <module>
    LayerId = cv2.dnn.DictValue
AttributeError: module 'cv2.dnn' has no attribute 'DictValue'

Modify file “/usr/local/lib/python3.10/dist-packages/cv2/typing/__init__.py”, comment out line 169 like following.[ref]

# LayerId = cv2.dnn.DictValue

CodeFormer

Error message:

"/work/stable-diffusion-webui/repositories/CodeFormer/facelib/utils/face_restoration_helper.py", line 7, in <module>
        from facelib.detection import init_detection_model
      File "/work/stable-diffusion-webui/repositories/CodeFormer/facelib/detection/__init__.py", line 11, in <module>
        from .yolov5face.face_detector import YoloDetector
      File "/work/stable-diffusion-webui/repositories/CodeFormer/facelib/detection/yolov5face/face_detector.py", line 20, in <module>
        IS_HIGH_VERSION = tuple(map(int, torch.__version__.split('+')[0].split('.'))) >= (1, 9, 0)
    ValueError: invalid literal for int() with base 10: '0a0'

This issue caused by pytorch.version will return illegeal number like 2.2.0a0+81ea7a4, so it will fail to parse this mesage, temperory solution is to remove the version number, edit file “stable-diffusion-webui/repositories/CodeFormer/facelib/detection/yolov5face/face_detector.py”. and modify line 20, IS_HIGH_VERSION to following version to following:[ref]

version_string = torch.version.split('+')[0]
version_string = ''.join(filter(str.isdigit, version_string))
IS_HIGH_VERSION = tuple(map(int, version_string.split('.'))) >= (1, 9, 0)

Now, the Stable Diffusion GUI should be working as well as expected

The post How to Install Stable Diffusion GUI on ARM64 Nvidia RTX platform appeared first on richliu's blog.

What truly impresses me is the board’s specifications. Not only does it boast a compact Micro-ATX form factor, but it also packs a powerful punch with four PCIe 4.0 x16 lanes, easily handling even demanding homelab tasks. The dual M.2 slots and dual OCULink port can enable software RAID for your NVMe disks. As a server board, it offers a BMC for remote control and system management. While the x64 platform remains popular, this board is an exceptional choice for those who prefer the non-x64 architecture, considering the limited choices with high speed fan options in server platforms.

Information

Important Spec

AsRock ALTRAD8UD-1L2T Spec web site

Where To Buy

US/Newegg

ASRock Rack ALTRAD8UD-1L2T Deep Micro-ATX Server Motherboard Single Socket (LGA 4926) Ampere Altra Max/Ampere Altra processors Dual 10G & 1G Without CPU,
Mother board only
USD$849.-

Asrock Rack Bundle ALTRAD8UD-1L2T Deep Micro-ATX Server Motherboard Single Socket (LGA 4926) with Ampere Altra Q64-22 64 cores & 2U passive cooler, Dual 10G – Integrated by Asrock Rack with CPU and heatsink
Mother board + Altra Q64-22 + 2U passive cooler(heatsink)
USD$1,500.-

UK/Server Factory Ltd

ASRock ALTRAD8UD-1L2T: Board only, Inc VAT: £870.00.

News

ASRock Rack ALTRAD8UD-1L2T Review This is the Ampere Arm Motherboard You Want

Cooling solution

Making Arm Desktops Viable Ampere Altra Noctua NH-D9 AMP-4926 4U and NH-U14S AMP-4926 (Preview tower fan system)
Ampere Altra/AltraMax AsRock ALTRAD8UD-1L2T Liquid Cooling System (my homemake Liquid/Water cooling system)

Other reference URL

ASRock Rack ALTRAD8UD Getting Started : A good note to start up to use the board.

Tricks for ALTRAD8UD

This chapter aims to guide users through using the ALTRAD8UD board, addressing potential information gaps left by the official ALTRAD8UD manual.

For memory compatibility reference, AsRock offers its own DRAM AVL (Approved Vendor List). Alternatively, users can access the Ampere Altra family device document, freely available without registration.

OpenBMC default username password

User: root
Pass: 0penBmc (first word is zero, not O).
both BMC console and GUI’s default username password are the same.

Serial Console over LAN Port

The board support Serial Port Header (9-pin COM1), it needs to use this serial port or VGA port to enter BIOS and get BMC IP.
After get the BMC IP, it can get more information from serial console over LAN port
Port 2201: ATF console
Port 2202: SCP console
Port 2203: Host console
Port 22: ssh console

Here is my script to run the sol and make it more easy to use

# It can put following two lines to bashrc
alias oit='ipmitool -U root -P 0penBmc -I lanplus -C 17'
alias ossh="sshpass -p 0penBmc ssh -o GlobalKnownHostsFile=/dev/null -o UserKnownHostsFile=/dev/null -oStrictHostKeyChecking=no"

# get host console 
ossh root@<bmc ip> -p 2203 
# get in bmc ssh console 
ossh root@<bmc ip> 

Serial Console Output To VGA

If you cannot see the serial console output to console port, probably it bacuase this option was enabled[ref], just turn it off and add console=tty0 or console=ttyAMA0 to grub.conf, ex:

GRUB_CMDLINE_LINUX_DEFAULT="console=ttyAMA0"

Ipmitool Fan Control [ref]

Manual Full speed
ipmitool raw 0x3a 0x01 0x64 0x64 0x64 0x64 0x64 0x64 0x64 0x64

Manual Half speed
ipmitool raw 0x3a 0x01 0x32 0x32 0x32 0x32 0x32 0x32 0x32 0x32

Auto fan control
ipmitool raw 0x3a 0x01 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0

NC-SI Port

In the user manual, NC-SI port is typo, the NC-SI port location is 5, in the document which marked as IPMI LAN port. if use this port, both host and BMC can use the same network cable to connect to the same ethernet.

Known Issues

Intel X550 massive AER error messages[ref1][ref2]

Seems that Intel NICs have compatibility issues with non-Intel platform systems. This can cause some server safety systems, like Advanced Error Reporting (AER), to generate many alerts and experience interface problems. A temporary solution is to add aspi_aspm=off to the Linux kernel parameters, ex:

GRUB_CMDLINE_LINUX_DEFAULT="console=ttyAMA0 pcie_aspm=off"

It will fix the AER issue.

The post AsRock ALTRAD8UD-1L2T Ampere Altra/AltraMax Note appeared first on richliu's blog.

E: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/pool/main/c/curl/libcurl3-gnutls_7.81.0-1ubu
ntu1.15_arm64.deb 503 Connection closed, check DlMaxRetries [IP: 192.168.1.254 3142]

The reason is the system file open limit, it needs to modify it, it needs to modify file /etc/security/limits.conf and add those lines

apt-cacher-ng soft nofile 102400
apt-cacher-ng hard nofile 102400

If your system is heavy loading server, it can increase those numbers, and not existed the system cat /proc/sys/fs/nr_open setting.

Write the limit nofile to /etc/init.d/apt-cacher-ng doesn’t work for me.

ref.
Unable to download anything from apt-cacher-ng proxy
Increase max open files for Ubuntu/Upstart (initctl)

The post apt-cacher-ng E: Failed to fetch 503 Connection closed, check DlMaxRetries appeared first on richliu's blog.

這次升級要從 x64 平台升級到 ARM64 平台，雖然目前 Linux 下 ARM64 支援度還是不錯，但是兩個平台能用的東西還是有差異，主要是紀錄一下升級過程中有什麼需要注意的事情

ARM64 Hardware Prepareation

CPU Board: Ampere Altra CPU + AsRock ALTRAD8U : 這個組合現在已經能在 newegg 上買到了，也是目前最強大的 ARM64 平台，當 homelab 平台非常適合

DRAM: 買這種主機板用的都是 RDIMM ，有可能便宜有可能貴，可以查 Gigabyte 的 QVL List[Ref] 內的 DRAM list 購買，有朋友就是在日本買到DDR4 2933 32GB RDIMM 一隻5000羊的DDR，目前匯率 NT$1,100.- 非常便宜

Network: AsRock ALTRAD8U 有 1*Intel i210 Ethernet 和 2*10G Intel x550 Ethernet ，這部份到不是問題，我另外拿了舊的 Intel x540 4 Port ethernet 當小 switch 用

SATA: 這種板子預設都沒有 SATA interface 了，所以要買 SATA 卡，有 PCIe 轉 SATA ，這個要注意 PCIe bandwidth 和 SATA 數量，一分錢一份貨。
如果要追求速度，淘寶上也可以買到便宜 SAS 外接 SATA ，淘寶的 LSI SAS 8port internal 卡大部份 NT$2500 到 NT$6000 都有，記得買轉接線要看好規格，SAS 接頭的規格很多，一不小心就會弄錯

SSD: 板子上介面有 2* M.2 2280，如果覺得不夠，可以買 OCU 轉 U.2 轉接線，或是還有 PCIe slot ，可以買 PCIe 轉 M.2 都可以。但是 U.2 的硬碟不一定好買，而且常常價位偏高，但是相對上是比較便宜的選擇

Software Move To ARM64 Platform

QEMU/Libvirt

這部份應該是最難的部份，不過理論上也不是不可以，我原來是 VirtualBox ，但是 ARM64 沒有 VirtualBox ，只有 Proxmox 有，剛看了一下，還支援 riscv64,loongarch64 [ref]，不過我個人不太想再學一套，所以還是用 virt-manager/libvirtd 這套，一般用途 virt-manager 和 libvirtd 應該也很夠用了。

目前用到現在都可以用，Ubuntu 22.04，Windows 10 for x86，Windows 11 for ARM，安裝沒什麼問題。只有 Windows 10 for x86 非常慢，慢到我不想再開了，不知道為什麼。

Qemu bridge devices: 這個應該是問題，如果是以前用 VirtualBox ，可以直接選 NIC card interface ，但是 virt-manager 似乎沒這個選項，所以要直接設定到 bridge device，不過影響不是很大，就只是設定一個 bridge device，然後將 host internet ethernet port device 綁上去就好了

其他可能是如果 Libvirt 要用 graphic 輸出，不能用 qxl ，只能用 VGA or ramfb ，尤其是 windows 只能用 ramfb，如果 Guest 是 Linux 可以用 VGA ，顯示比較正常，如果連 GUI 都沒有，就直接看 console 輸出吧，問題更少

Qemu file sharing

QEMU 支援 virtiofs ，不過這個功能在 gentoo 內 ARM64 是不能用的，因為不能用，所以我也不知道 x64 那邊會有什麼限制
另外一個是 virtio-9p ，這個缺點是分享進 VM 之後，沒辦法使用 user 權限存取，用 root 身份限制也很多，像是存取 non-root user 檔案會有問題[ref]

有其他需求的人可以考慮改用 samba or nfs ，但是效能就會大大下降了，這點可能是有效能需求的人要考量的

Firewall

原來的 Firewall script 似乎沒辦法和 Libvirt 一起工作，我又不想修改，最後比較簡單的作法就是在我的 script 執行完之後，再重啟 libvirtd ，這樣就正常了。
正常後別忘了執行 /etc/init.d/iptables save 儲存，這樣重開機之後才會正常執行 firewall rule.

Docker

試了一下似乎沒什麼問題，但是目前沒什麼要跑的 APP ，所以應該測不出什麼
不過之前 arm64 k8s docker [ref]都跑過了，應該踩雷的機會低很多

Other Software

在 Linux 下轉移到 ARM64 平台問題不大，大部份的 software 都是安裝之後 copy 過去
就算是在系統上有資料的 mysql 和 apache 都是這樣

這次我用了一個方式轉移 configuration files
1. 將要轉移的程式設定檔 commit 到 git server
2. 從新機器將 git server 拉回來 configuration files
3. diff 之後將新的設定套上去

因為我是 gentoo 玩家，gentoo 升級 packages 常常需 merge configuration files 了，所以這個轉移的過程不難，只是很耗費時間而已。不過因此絕大部份的設定檔都沒有問題，反而是少數系統本身自帶的設定檔反而怪怪的

KDE

之前我都遠端跑 vnc + kde ，不過 Gentoo 有個 BUG 就是 plasmashell 的 CPU usage 會很高
這次換 virt-manager 之後可以用 ssh remote control VM，發現這個方式還不錯，反正我也只有用到這個功能，KDE 等有需求的時候再開 vnc 就好了，平時是不需要開的

搬移步驟

1. 備份系統碟，以備不時之需
2. 重新規劃電腦的網路卡，如果只有一張就直接設成 bridge interface
3. software raid 的部份可以參考自己之前的文章 Linux Software Raid 系統轉移
4. UEFI 的部份可以參考自己之前的文章 Gentoo ARM64 UEFI Install Guide
5. 搬移 APPs
5.1 重要的 APP 才要搬資料，ex: Web, mysql
5.2 不重要的 APP 只要搬設定檔，ex: apt-cache-ng, squid 等等
5.3 如果是切換平台，要確定 VM 沒問題才能切

Known Issues

IPv6 Default Router

不知道為什麼，在 wan 的 bridge interface 會增加一個 default route fe80::1 ，這個設定會讓 IPv6 失效，後來解決方式是在 LAN 端加上一個 IP address fe80::1 才解決

efi boot

因為我系統碟是做 software raid ，每次編完 kernel 再 update grub 時，記得 efi boot 時有時會 fail ，試著將兩顆 nvme 都 mount 上來加上再安裝一次 efi partition 似乎才會正常，找了一下沒找到原因，不管了，反正能動就好

grub-mkconfig -o /boot/grub/grub.cfg
umount /boot/efi
mount /dev/nvme0n1p1 /boot/efi
grub-install --target=arm64-efi --efi-directory=/boot/efi --bootloader-id=Gentoo
umount /boot/efi
mount /dev/nvme1n1p1 /boot/efi
grub-install --target=arm64-efi --efi-directory=/boot/efi --bootloader-id=Gentoo
umount /boot/efi

ref.
None

The post [Gentoo] Upgrade From x64 to ARM64 appeared first on richliu's blog.

Chinese version Article is here (Ampere Altra/AltraMax 加AsRock ALTRAD8UD-1L2T 水冷方案)
2024/01/31 Update: Ampere will have 2U passive cooler Making Arm Desktops Viable Ampere Altra Noctua NH-D9 AMP-4926 4U and NH-U14S AMP-4926

The Ampere Altra/AltraMax CPU features a maximum of 128 ARM64 cores, typically found in cloud server. However, cloud server Care not suitable for placement in offices or homes due to the high-speed fans used for heat dissipation. The noise generated by these fans is generally intolerable for regular users. Additionally, adapting server motherboards to standard consumer motherboards is inconvenient. If there is a desire to utilize such a CPU in a different setting, the ADLink Ampere Altra Dev Platform presents a viable solution, complete with the option for liquid cooling.

Another solution is the AsRock ALTRAD8UD-1L2T, currently available on Newegg. Paired with an Ampere Altra Q64-22 CPU (64 cores, 2.2GHz), it comes with a complimentary 2U passive heatsink, priced at US$1,500, which can be considered affordable. However, it’s worth noting that the included heatsink is passive. If you prefer liquid cooling over installing a fan on the passive heatsink, then this article is the right guide for you.

The author have adding a declaration at the beginning of articles that may pose potential risks:

• This article is not an official endorsement by Ampere, Newegg, or any other mentioned merchants or vendors. It is solely a personal record.
• The author has a business relationship related to the production of the mentioned motherboard.
• The author is a first-time to use liquid cooling computer system , and the content may not be entirely accurate. Please be aware of potential risks.
• This article is not an official document from Ampere or AsRock.
• The original article is in Chinese, and the English translation is done using machine translation. Please forgive any inaccuracies in wording.
• I am not responsible for any consequences resulting from this article; for example, losses incurred due to improper implementation of instructions provided in this article.

The Ampere system primarily focuses on Cloud-native CPUs, and as of now, there is no manufacturer producing general-purpose cooling systems for it. The included 2U passive heatsink in this system is designed for server use, utilizing high-speed server fans for heat dissipation. While the Q64-22 included by Newegg has lower power consumption, its only cooling method involves direct airflow from an electric fan. Alternatively, users may resort to attaching an 8-inch fan onto it. Such cooling solutions are deemed unacceptable for enthusiasts and professionals like myself.

If you intend to upgrade the cooling system, typical tower-type coolers require replacing the fixture on the back of the motherboard. Moreover, compatibility with the fastening mechanisms of this specific motherboard can be challenging to find. Considering that only front-facing screws can be used, the only viable cooling solution is a liquid cooling system. Therefore, I have been experimenting with a liquid cooling solution that works for this scenario. I’m sharing this information for those adventurous individuals who might encounter similar challenges, and perhaps this approach can inspire some creative solutions.

In the upcoming sections of this article, I will guide you on how to set up a liquid cooling system specifically tailored for the Ampere Altra on the ALTRAD8UD-1L2T motherboard. In theory, since the screw hole positions are the same, this method should be applicable to other Ampere motherboards as well. However, the focus of this content will primarily be on the ALTRAD8UD-1L2T. The article will also cover any pitfalls encountered during assembly and the items needed for the setup(Chinese website).

Preparation

ALTRAD8UD-1L2T

You can find the specifications for this motherboard online, and I won’t repeat them here. However, what might confuse some people is the term “Deep mATX.” This is a specification unique to AsRock, where they increase the depth dimension of the mATX form factor while maintaining the standard width. The purpose is to provide more space to accommodate additional components. This particular specification is commonly seen in AsRock’s server-grade motherboards.

Chassis

Generally, most computer cases should work. However, I made a mistake on my end by choosing a case that was too small. The case I purchased is the Antec P110 Silent. While it has liquid cooling mounting points, it is not suitable for adding a liquid cooling system. The internal space is too limited, lacking a designated area for securing the water pump. Additionally, the Antec P110 liquid cooling radiator cannot be placed at the top as it interferes with the motherboard. Consequently, the radiator had to be positioned at the front of the case.

When making a purchase, it’s crucial to choose a case that is suitable for water cooling. In more extreme cases, considering a caseless system might be an option (hopefully, this can be updated in the next release).

Cases designed for liquid cooling systems typically have more significant internal space to accommodate a water pump, and there is ample room for fans without obstruction. Some all-in-one water cooling designs may be more compact and might not encounter this issue.

Liquid Cooling System

Liquid cooling systems have some considerations to take into account:

• Tubing type: Choose between soft or hard tubing. Hard tubing is suitable for experienced users, offering a longer lifespan and a more aesthetic appearance. Soft tubing is easier for assembly, making it suitable for beginners.
• All-in-one (AIO) or custom loop: Decide whether to go for a pre-assembled AIO liquid cooling system or build a custom loop system.

Since the initial focus was on researching liquid cooling, the combination chosen was a custom loop with soft tubing. Building a custom loop may encounter some issues, with one of the main challenges being the uncertainty about specifications.

The first crucial aspect is determining the size of the liquid cooling tubing, as additional components will depend on this size, limiting the options for procurement. Currently, the mainstream sizes are two-eighths (2/8’), three-eighths (3/8’), and four-eighths (4/8’) liquid cooling tubing.

Another parameter is the thickness of the tubing, thick tubing and thin tubing.
* thick tubing refer to 10/16mm or 3/8”-5/8” tubing.
* thin thbing refer to 10/13mm or 3/8”-1/2″ tubing.

Most common the inner diameter specification is G 1/4, when find some liquid colling componens like pressure relief valve needs it.

Once you’ve confirmed your inner and outer diameter, you can proceed to purchase the necessary liquid cooling components.

At a minimum, you’ll need:

• CPU water block *1
• Water Pump *1
• 240mm radiator *1
• 12cm fans *2
• Liquid coolant 500cc ~ 1000cc
• Tubing 2M
• Thermal paste
• Pressure relief valve
• Fittings: Quick connectors *6, 90-degree rotary fittings *(1~2)
• Tools for filling and draining: Filling tool, draining tool (watering can, large syringe, drainage hose)

Optional components:

• Flow meter, temperature gauge (note: buying these requires purchasing two additional quick connectors).

In practical testing, if focusing solely on CPU cooling, it appears that a 240mm radiator is sufficient. The thin three-eighths (3/8”) tubing provides a large enough water flow diameter. Choosing a smaller tube diameter may not be as suitable for this purpose.

CPU water block

This turned out to be quite troublesome. My original plan was to purchase a 50mmx50mm heatsink and connect it to the heatsink on the right. Then, I intended to secure it to the CPU socket using the fixtures shown in the image. However, as expected, unexpected issues arose – the world isn’t always that kind.

1. There was no water block with a 50mmx50mm connector available. So, I ended up buying a 40mmx40mm connector, one size smaller. This resulted in poorer heat dissipation performance. Fortunately, I never had the chance to test the inferior heat dissipation, so this step failed unnoticed.
2. Here’s where I made a mistake. Custom water block connectors, like the one shown, are typically 2/8″ fittings. However, the main liquid cooling system I bought has 3/8″ fittings, making them completely incompatible. The options were either to change the fittings or change the entire system. Changing the fittings was the more cost-effective choice, so I opted for that.

After connecting , it looked like the image on the right – and just looking at it made me worry that water would leak out.

So, I bought a two-eighths(2/8″) tubing to connect to this connector. However, while it could connect, it couldn’t link up with my three-eighths(3/8″) tubing in the water cooling system. I had to reluctantly abandon the custom water block connector.

The final solution had nothing to do with this connector.

Communical CPU water block

During the installation, I discovered that the Bykski CPU-XPH-T8 CPU water block, along with the provided adapter bracket, could be securely attached to the Ampere Altra’s socket. This was fantastic news and saved me a great deal of effort.

Although it’s designed for Intel LGA1700/LGA2066 sockets, the flexibility in its hole configuration allowed for adjustments, providing me with the necessary adaptability. As for whether other CPU brackets might be compatible, it’s likely that some could work. However, I haven’t had the chance to try others yet. If you have attempted this and found success, please let me know.

The base of this water block is quite substantial, and my intuition tells me that the cooling performance should be excellent. Therefore, I didn’t explore other compliant CPU water blocks.

However, another issue arises. The water block is 56mmx56mm, whereas the width of the Ampere Altra CPU socket is 55mm. The water block does not fit. At this point, there are two possible solutions:

Upon observing the CPU’s structure, the first solution seemed unfeasible. Cutting down the thickness of this water block would compromise its structural integrity, not to mention finding the tools or a professional service for the task.

Thus, the only viable option is the second solution. The simplest and most intuitive way to implement this is by using a thermal pad made of silicone to transfer heat upwards from the CPU.

Silicone Thermal Pad

But things couldn’t be as simple as I, a fool, initially thought. In conclusion, regular silicone thermal pads are not suitable for power monsters like the Ampere Altra. This type of thermal pad may be effective for low-power devices such as routers, switches, NVMe disks, etc., but it’s not useful for high-power CPUs.

In the image, there are 2mm thick silicone thermal pads, and you need two layers because the height is just that much. Essentially, this provides only a slight improvement over having no heat dissipation at all, so it’s not worth considering.

As for the high-efficiency silicone thermal pads available in the market, I don’t know if they would work. If you have them on hand, you can give them a try. However, I believe the chances of success are quite low.

Purple Copper Thermal Pad

Nowadays, a common method for cooling is using purple copper heat pipes. This type of heat dissipation is quite effective, and the 2U passive heatsink included with Newegg utilizes these heat pipes. Therefore, if we can customize the heatsink, wouldn’t that solve the problem? Luckily, the country right next to my country is the world’s largest manufacturing factory, capable of customizing just about anything. This made the solution very affordable and instantly resolved my problem.

The dimensions of the purple copper thermal pad are: Length x Width x Height = 64x54x4mm, with a rounded R5 edge.

Alternatively, you can measure the dimensions of the piece under the 2U passive heatsink to get the specifications. Placing it on the original heatsink, I can say the dimensions match perfectly. The craftsmanship is quite impressive, and there are no issues when it’s actually placed.

Assembly

First, place the purple copper thermal pad, and remember to apply thermal paste before doing so.

Apply thermal paste before placing CPU water cooling block.

Assemble the components.

Actually, I assembled the tubing on the left side incorrectly. I’ll need to purchase materials and quick connectors to replace it when I have the time. But for now, let’s leave it as is and make do.

Overall view.

This way, it’s evident that the case isn’t ideal. It’s too cramped, affecting airflow, and it cannot accommodate the components properly.

Shopping

If you are interested in replicating my setup, all the components were purchased from Taobao. I spent extra for testing and experimenting, but in reality, you should be able to set up the entire cooling system for around US$100 or NTD$3,000. This is comparable to the cost of an efficient dual-fan tower cooler.

Since this is a Chinese website, I’ll keep the text in Chinese and provide English translations afterward.

Liquid Cooling System

watertiger定制台式机电脑分体式CPU显卡水冷软管套装冷散热器 (“WaterTiger Custom Desktop Computer Split-Type CPU GPU Liquid Cooling Soft Tube Kit Radiator”)
Enter the URL choice “單 CPU 水冷 240 冷排” (“Single CPU Liquid Cooling 240 Radiator”)

At first, I chose this brand because the photos looked appealing, but it turned out to be a fortunate coincidence that saved me a lot of exploration time. This brand integrates various components into a set for sale, and the components they use are of decent quality. For example, they use the Enermax NEOChanger as the water pump, which is also a good pump. If you are familiar enough, you might not need these kits.

This kit does not include a pressure relief valve. You’ll need to purchase a pressure relief valve separately, and depending on your needs, you might want to buy additional quick connectors

Clamping Fittings

All the components can be purchased from the same place, saving the hassle of searching everywhere. It’s a good idea to buy a few extra parts to avoid trouble later on. The time cost is much higher than the cost of the components.

电脑水冷头扣具卡具CPU显卡南北桥固定笔记本散热器压条五金夹具 (“Computer Water Cooling Head Fastening Fixture, CPU GPU North-South Bridge Fixing, Laptop Heatsink Clamp, Hardware Clamp”)

Choice 台式机CPU冷头扣具(“Desktop CPU Cooling Head Fastening Fixture”) *2

M2 M3 M4不锈钢平头盘头螺丝钉水冷头水冷排机箱风扇安装长螺丝 (“M2 M3 M4 Stainless Steel Flat Head Countersunk Screws for Water Cooling Head, Radiator, Case Fan Installation – Long Screws”)

Fastening fixtures for securing CPU coolers and water blocks. The components can be increased based on requirements. The minimum required quantity includes
M3手拧螺母 (M3 thumb nuts)*4，M3*30 (M3 screws) *4 ，M3垫片(M3 washers) *4 ，M3六角螺母(M3 hex nuts) *4

电脑水冷水流计 液冷散热器流量器 流体温度观测器透明水流指示计 (“Computer Liquid Cooling Flow Meter, Liquid Cooler Flow Indicator, Fluid Temperature Monitor Transparent Flow Meter”) (option)

Choice: 3孔水流观测器 + 温度计 (“3-Port Flow Meter + Thermometer for Water Flow Observation”)

Nice to have items, but it can make me feel better.

Test Result

This system feels relatively quiet and not noisy.

The following tests are only for verifying the liquid cooling setup and are not formal performance tests. The results are provided for liquid cooling reference and do not include testing power consumption, etc.

Room temperature: 25 degrees Celsius
CPU : Ampere AltraMax 128-30
Run command : stress-ng –cpu 128 –iomix 128 –vm 128

Gentoo Linux kernel 6.6 ，CPU temperature is approximately，Power consumption CPU 150W/ IO 19W
Ubuntu 22.04 kernel 6.2(?)，CPU temperature is approximately 72 degrees，Power consumption CPU 178W/ IO 20W

In conclusion, I hope this article has been helpful for those eager to try it out.

Afterword

Newegg has also listed another model, which is only the motherboard version, only US$849.-
ASRock Rack ALTRAD8UD-1L2T Deep Micro-ATX Server Motherboard Singel Socket (LGA 4926) Ampere Altra Max/Ampere Altra processors Dual 10G & 1G

The post Ampere Altra/AltraMax AsRock ALTRAD8UD-1L2T Liquid Cooling System appeared first on richliu's blog.

English version article is here (Ampere Altra/AltraMax AsRock ALTRAD8UD-1L2T Liquid Cooling System)
2024/01/31 Update: Ampere will have 2U passive cooler Making Arm Desktops Viable Ampere Altra Noctua NH-D9 AMP-4926 4U and NH-U14S AMP-4926

Ampere Altra/AltraMax CPU 最高單顆 CPU 是 128 核 ARM64 的 CPU，一般都是在伺服器主機上，但是伺服器主機非常不適合放在公司內或是家用，因為一般伺服器主機都會用高轉速風扇散熱，一般人是不能忍受這樣的噪音。伺服器主機板也不方便改裝到一般家用主機板，如果真的想要用，ADLink Ampere Altra Dev Platform 是一個不錯的解決方案，也有水冷的選項。
另一個解決方案就是 AsRock ALTRAD8UD-1L2T ，目前在 newegg 上架了，配合一個 Ampere Altra Q64-22 (64 cores 2.2G) 的 CPU，附贈一個 2U Passive 的散熱片，價格 US$1,500.- ，應該算便宜。但是送的是被動散熱，如果不想要裝一顆風扇到被動散熱上，想要改成水冷散熱，那看我這篇文章就對了。

本人習慣，在有可能造成危險的文章，會在文章前面加上宣言
* 本篇文章並非 Ampere, Newegg 或是其他任何文章內提到的商城及店家業配文，純屬個人紀錄
* 本文作者和部份這塊板子的生產有利益關係，特此揭示
* 作者是第一次接觸電腦系統水冷，內容可能不盡正確，請自行注意風險
* 本文並非 Ampere 或是 AsRock 官方文件
* 本人不對本篇文章的造成的任何影響負責任；例如看本人文章操作不當造成的損失等等

Ampere 系統主要是專攻 Cloud nativate CPU，所以直至目前為止，並沒有廠商生產一般用途的散熱系統。這個系統附贈的 2U Passive 散熱片是 Server 專用的，使用的是 Server 高速風扇散熱；雖然 newegg 附的是 Q64-22 ，功耗比較低，但是唯一的散熱方式就是拿電風扇直接吹，要不然就是卡一個8吋風扇在上面，這種散熱方案像我這樣的專業玩家是無法接受的

如果要進階散熱，一般的塔型散熱系統需要將主機板背面的夾具換掉，而且也要能跟這個板子的扣具相容，找到相容的塔型散熱難度可能也不小。考量到只能在正面鎖螺絲，那唯一能用的散熱系統就是水冷式散熱。所以我這段時間摸索出了一條可以用的水冷散熱解決方案，在此分享給想要踩雷的朋友們，或許這個做法也可以激發各位的創意。

接下來會在這篇文章內介紹要如何在 ALTRAD8UD-1L2T 板子上，搭建屬於 Ampere Altra 的水冷系統，理論上因為螺絲孔位一樣，所以 Ampere 其他的板子也可以用，但是本篇內容主要以 ALTRAD8UD-1L2T 為主，而本文亦會提到踩過的地雷或是採購項目。

準備

ALTRAD8UD-1L2T

上網可以找到這塊板子的規格，這些就不再重覆，但是一般人會比較迷惑的是，什麼是 Deep mATX ？
這個是 AsRock 自己的規格，是增加 mATX 的深度尺寸，保持 mATX 的寬度規格，以增加更多空間容納更多元件。這個特別的規格常見於 AsRock server grade 的主機板上。

機殼

一般機殼應該都可以。不過我這邊犯了一個錯誤，機殼買太小，我買的機殼是 Antec P110 Silent ，雖然有水冷孔位，但是這個機殼不適合加裝水冷，內部空間太小，沒有固定水泵的地方，Antec P110 水冷排無法放在上方，會卡到主機板，所以最後水冷排是放機殼前方

採購時，請找適合水冷的機殼，更極端一點可以考慮無機殼系統（希望下次更新可以更新無機殼系統）。

水冷機殼內部空間需求更大，要有地方放水泵，還有風扇放下去不會卡住。
有些一體式水冷的設計會比較小，比較不會有這個問題

水冷設備

水冷散熱有一些需要考慮的
* 軟管或是硬管（硬管比較適合有經驗的，壽命長，較美觀；軟體組裝方便，適合新手）
* 一體式水冷（廠商已經組裝好）或是分體式水冷（自組水冷系統）

因為一開始是研究如何建置水冷，所以一定是分體式水冷加軟管的組合。自組水水冷又會碰到一些問題，最主要的是不知道規格為何？
第一件需要確定的是水冷管大小，因為加購設備就是看水冷管大小，這會限制採購品項
目前主流有 二分(2/8’)、三分(3/8’)、四分(4/8’)水冷管，但是實際上在找產品的時候，會標為像 6*10(二分) 9.5*12.7(三分，有些會標 10*13) 另外還有 8*12 不知道對應到什麼，數字小的是內徑，數字大的是外徑 。三分又有分三分薄和三分厚，三分薄就是 10/13 ，三分厚就是 10/16 ，如果不是老手很容易在這邊搞錯。本文用的全部都是三分薄的系統

以上有英制和公制，要特別小心注意規格，像洩氣閥內徑規格就要用 G 1/4 規格的（目前大都是此規格）
確定好自己的內外徑之後，就可以採購需要的水冷套件

最少需要的是
CPU水冷頭*1
水泵*1
240寬冷排*1
12CM風扇*2
水冷液 500cc ~ 1000cc
水管 2M
散熱膏
洩壓閥
接頭：快速接頭*6 ，90度旋轉彎頭*(1~2個)
注水工具，放水工具（加水壺，大針筒，放水管）
可選：
流量計，水溫計（註：買這個要多買二個快速接頭）

實測如果只針對 CPU 散熱，目前看起來 240 水冷排就夠，而三分薄提供了夠大的水流口徑。
如果管徑再小可以不太合適。

CPU水冷散熱頭

這個頗麻煩的，我原來的規劃是買一個 50mmx50mm 的散熱片接到像右邊的散熱頭上。然後用圖中的夾具固定在 CPU 座上面，但是不出意外的就出了意外，這世界上沒這麼好的事情

1 ) 沒有 50mmx50mm 的水冷頭接頭，所以買的是 40mmx40mm 的接頭，小了一圈，散熱性能較差，幸運的是我根本沒機會測到散熱性能較差這步就失敗了

2) 我在這邊犯了一個錯誤，這種像是定製的水冷接頭都是 2/8″ 頭，但是我買的主要水冷系統是 3/8″ 的系統，完全接不上去，要就是要換頭，或是換系統；當然換頭比較便宜，所以選擇換頭

接上去之後像是右邊這樣，這光看都覺得水會跑出來吧

所以我買了二分管想要接這個接頭，但是二分管是可以接上去沒錯，但是接不上我水冷的三分管系統。只能果斷放棄接定製水冷頭

最後的解法已經和這個接頭無關了

商用水冷頭

在裝的時候發現買的套件附的 Bykski CPU-XPH-T8 這顆 CPU 水冷頭加轉接夾具可以合上到 Ampere Altra 的底座，這簡直太棒了啊。省了非常多功夫

雖然是給 Intel LGA1700/LGA2066用的，所以它的孔位設計成非常彈性，這就給了我調整的空間。至於其他 CPU 的夾具有沒有可能可以用，應該也有可以用的，不過這邊我還沒機會試，如果你有試過可以告訴我。
而這個水冷頭的底座非常大，直覺告訴我散熱效果應該非常好，所以也並未去找其他合規的 CPU 水冷頭

但是接下來又有另一個問題，這個水冷頭是 56mmx56mm ，而 Ampere Altra CPU socket 寬度為 55mm，這個水冷頭放不下。這時有二個選擇方案
1) 切割水冷頭讓它變小（還沒考慮高度問題）
2) 將熱從 CPU 接到水冷頭

第一個方案觀察了一下 CPU 工藝，覺得不可行，因為這顆水冷頭的厚度切下去會影響結構，更別提還要想辦法找到工具或是店家切割。

那就只能用第二個方案了，這個方案最簡單並且非常直覺的方式就是用矽膠導熱墊將熱拉上來

矽膠導熱墊 (thermal pad silicone)

但是事情怎麼會像我這個笨蛋想的這麼簡單，先說結論，一般的矽膠導熱墊不適用於 Ampere Altra 這種 Power Monster ，所以完全不用考慮。這種矽膠導熱墊給小功率的 router/switch/nvme disk 等等或許有用，對大功率的 CPU 是沒有用的

圖中是 2mm 厚的矽膠導熱墊，而且要二層，因為高度就是這麼高，基本上這個只比不接散熱好一點點而已，所以別想了

至於市面上賣的高效率矽膠導熱墊有沒有用我就不知道了，如果手上有現成的可以試試，不過我認為成功機率很低

紫銅導熱片

那現在一般的散熱是怎麼散熱呢？一般都是紫銅導熱管，這個散熱效率還不錯，newegg 附贈的 2U passive heatsink 就是使用這種導熱管散熱。所以如果可以客製散熱片，這樣不就解決問題了嗎？幸好我們旁邊就是世界最大的製造工廠，任何怪東西都可以客製，這樣一片非常便宜，瞬間解決了我的問題

紫銅導熱片的尺寸規格是
長x寬x高 = 64x54x4mm ，另外要倒 R5 的角

也可以量 2U passive heatsink 下面那一塊得到規格，放在原廠的散熱片上只能說尺寸一樣
工藝還真不錯，實際放下去也沒有問題

組合

先放上紫銅導熱片，記得放之前塗散熱膏

放上去之前再塗散熱膏

合體

其實這張圖左邊的管子我裝錯了，有空要再去買材料和快速接頭來換，但是現在就先這樣吧，撐一下。

全貌

這樣就看得出來這機殼不好了，太擠太小影響風流，不能放在上面

購物

如果你是想學我的方案，我所有的零件都是從 Taobao 購入，我因為有測試還有多花錢
但是實際上應該 US$100, NTD$3,000 左右可以搞定整個散熱系統。
跟一個效率好的雙風扇塔扇差不多

水冷系統

watertiger定制台式机电脑分体式CPU显卡水冷软管套装冷散热器
進去後選單 CPU 水冷 240 冷排

當初是看照片比較好看所以買這家的，但是誤打誤撞節省我很多摸索的時間。
這家也是將各家設備整合弄一套出來賣，但是東西用的還不錯，像水泵用的是 Enermax NEOChanger ，也是還不錯的水泵。如果夠熟是可以不用這些套件。

這個套件沒有洩壓閥，另外要多採購洩壓閥，還有依需求可以多買快速接頭

夾具

以下都在同一家採購，不用到處找，零件可以多買一點，免得到時候要找麻煩，時間成本遠高於零件成本。

电脑水冷头扣具卡具CPU显卡南北桥固定笔记本散热器压条五金夹具

選台式机CPU冷头扣具 *2

M2 M3 M4不锈钢平头盘头螺丝钉水冷头水冷排机箱风扇安装长螺丝

固定 CPU 冷頭扣具和水冷頭的零件，這個可以依需求增加，以下是最低需求量
M3手拧螺母*4，M3*30 *4 ，M3垫片*4 ，M3六角螺母*4

电脑水冷水流计 液冷散热器流量器 流体温度观测器透明水流指示计 (option)

選: 3孔水流观测器 + 温度计

可有可無的東西，不過有比較好玩

測試結果

這套系統個人感覺相對安靜，不會吵

以下測試只是驗證水冷測試，並非正式的 performance test ，結果僅供建置水冷參考，並非測試 power consumption等結果

室溫 25 度
CPU : Ampere AltraMax 128-30
Run command : stress-ng –cpu 128 –iomix 128 –vm 128

Gentoo Linux kernel 6.6 ，CPU 溫度約在 67 度，Power consumption CPU 150W/ IO 19W
Ubuntu 22.04 kernel 6.2(?)，CPU 溫度約在 72 度，Power consumption CPU 178W/ IO 20W

最後希望這篇文章有幫到躍躍欲試的你

後記

Newegg 還有上架另一個型號，只有主機板的版本，只賣US$849.-
ASRock Rack ALTRAD8UD-1L2T Deep Micro-ATX Server Motherboard Singel Socket (LGA 4926) Ampere Altra Max/Ampere Altra processors Dual 10G & 1G

The post Ampere Altra/AltraMax 加AsRock ALTRAD8UD-1L2T 水冷方案 appeared first on richliu's blog.

1. 先下載這個 use_existing_session.patch patch file ，先上好 patch

2. 執行完之後

執行以下命令並重新啟動 chrome remote desktop service

export FIRST_X_DISPLAY_NUMBER = 0
echo "0" > ~/.config/chrome-remote-desktop/Xsession
systemctl restart chrome-remote-desktop@"your_user".service

目前在 Dell Notebook + Ubuntu 22.04 沒有問題，雙螢幕也可以支援

The post Ubuntu 22.04 + Chrome Remote Desktop Use Current Session appeared first on richliu's blog.

The reason for this issue is that the ‘renderer: NetworkManager’ line is missing from the network YAML configuration file. To resolve this problem, you should edit your YAML file and add the following line.

renderer: NetworkManager 

to /etc/netplan/00-installer-config.yaml (my example), then restart network manager will fix it.

ref.

https://ubuntuforums.org/showthread.php?t=2473217
https://ubuntu.com/core/docs/networkmanager/networkmanager-and-netplan

The post Kubuntu NetworkManager No Available Network appeared first on richliu's blog.

Just disable it in your BIOS and add “console=tty0” to /etc/default/grub, then update grub.cfg. it will work.

In AMI BIOS, it should be in the Advanced->Serial Port Console Redirection->Console Redirection

Set it to [Disabled]

Some Linux distrubtions might not use GRUB_CMDLINE_LINUX in the file /etc/default/grub with grub paramater GRUB_ENABLE_BLSCFG=true, if you this parameter in th e grub.cfg file, please disable it.

ref.
[RFC PATCH v2 0/3] Prefer working VT console over SPCR and device-tree chosen stdout-path

The post ARM64 cannot output Linux kernel message to VGA appeared first on richliu's blog.

• * The Ubuntu/Fedora rootfs on tftp/nfs server:
  It can refer to the my article MIPS/ARM64 Debian/Ubuntu debootstrap for building the Ubuntu rootfs.
  If want to build the Fedora rootfs, it can install a new system on qemu and copy it from the qcow2 image.

• * PXE Server:
  You can refer my article ARM64 Centos8/Ubuntu PXE Boot Install on Ubuntu Server to build a PXE server.

• NFS Server:
  Assume the rootfs for Ubuntu is on /nfs/ubuntu2204 and Fedora 36 is on /nfs/fedora36.

Ubuntu PXE Boot

Copy vmlinux and initramfs to tftp server. Copy your current system’s vmlinux to your tftpboot. For example, create a directory for Ubuntu 22.04 in /tftpboot/nfs/ubuntu2204, and copy the vmlinuz to the folder using the following command:

$ cp /boot/vmlinuz-$(uname -r) /tftpboot/nfs/ubuntu2204/vmlinuz

Copy the initrd.img to the folder using the following command:

$ cp /boot/initrd.img-$(uname -r) /tftpboot/nfs/ubuntu2204/initrd.img

Setup your PXE server’s grub.cfg and add a record. Use the following example for the record:

menuentry 'Ubuntu22.04 nfsboot' { 
    linux nfs/ubuntu2204/vmlinuz ip=dhcp root=/dev/nfs nfsroot=<NFS server IP>:/nfs/ubuntu2204,nolock rw netboot=nfs 
    initrd nfs/ubuntu2204/initrd.img 
}

Fedora PXE Boot

The procedure is similar to Ubuntu, but the default Fedora initramfs doesn’t support nfs boot.
1. Rebuild the initramfs with dracut. You need to add nfs and network support to your Fedora initramfs using the following command:

$ dracut --add nfs --add network --force

It will generate the same name vmlinuz and initramfs. Copy it to your directory just like Ubuntu.
2. Disable selinux.
Default Fedora selinux is enabled, but it cannot enable on nfs rootfs (or it can but needs to set up something). You can disable it by adding selinux=0 on boot kernel parameter. Use the following example for the record:

linux nfs/fedora36/vmlinuz boot=nfs root=/dev/nfs nfsroot=<NFS server IP>:/nfs/fedora36,nolock rw netboot=nfs ip=dhcp selitarnux=0

The post Ubuntu/Fedora PXE Boot appeared first on richliu's blog.