Ragflow on ARM64

RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine built on deep document understanding. It is very easy to use and install. However, RAGFlow does not officially support the ARM64 platform, so it needs to be built from source for deployment on ARM64 systems.

In this article, I will skip other steps such as installing Ollama.

System Specifications:

• OS: Ubuntu 22.04 on an ARM64 virtual machine
• CPU: Ampere Altra
• DRAM: 32GB (allocated to the VM, though smaller models do not require this much memory)

At the time of writing, Infinity does not support the ARM64 platform, but that is not an issue since we do not need it. RAGFlow uses Elasticsearch as its default AI database engine, and Elasticsearch does support the ARM64 platform.

Building RAGFlow on ARM64

Build ragflow command [ref. Build a RAGFlow Docker Image]

git clone https://github.com/infiniflow/ragflow.git
cd ragflow/
docker build --build-arg LIGHTEN=1 -f Dockerfile -t <username>/ragflow:<version> .

docker build --build-arg LIGHTEN=1 -f Dockerfile -t user/ragflow:v0.16.0 .

If everything is set up correctly, the Docker images will display an output similar to the following:

docker images
REPOSITORY                                      TAG                            IMAGE ID       CREATED         SIZE
user/ragflow                                    v0.16.0                        8a71ac9cb2fa   3 hours ago     5.73GB

Note that this image does not include pre-built models. However, models can be installed separately using other methods.

Running RAGFlow

Before running the docker-compose command, assign the RAGFlow Docker image name to the docker-compose configuration.

export RAGFLOW_IMAGE=user/ragflow:v0.16.0
docker compose -f docker/docker-compose.yml up -d

If everything is set up correctly, you should see a message similar to the following:

[+] Running 10/10
 ✔ Network docker_ragflow      Created                                                                         0.2s
 ✔ Volume "docker_minio_data"  Created                                                                         0.0s
 ✔ Volume "docker_redis_data"  Created                                                                         0.0s
 ✔ Volume "docker_esdata01"    Created                                                                         0.0s
 ✔ Volume "docker_mysql_data"  Created                                                                         0.0s
 ✔ Container ragflow-mysql     Healthy                                                                        21.7s
 ✔ Container ragflow-minio     Started                                                                         1.1s
 ✔ Container ragflow-redis     Started                                                                         1.1s
 ✔ Container ragflow-es-01     Started                                                                         1.1s
 ✔ Container ragflow-server    Started                                                                        22.3s

Downloading Models with Ollama

Ollama needs to download certain models for system functionality. Here’s an easy way to download the models—you can choose the ones you prefer.

ollama pull deepseek-r1:14b
ollama pull deepseek-r1:32b
ollama pull smartcreation/bge-large-zh-v1.5:latest

After downloading the Ollama models, you need to add them in the “USER → Model Providers → Add Ollama Model” section. Below is an example of how to add the Deepseek model. Once you understand the process, you can add more models as needed.

At a minimum, you will need one chat model and one embedding model.

After pressing OK, you can configure the System Model Settings, which should look something like this:

Using RAGFlow

Now, you are ready to use the Knowledge Base and Chat features to explore RAGFlow.

For example, you can ask the engine to generate an ARM SIMD sample code based on the ARMv8 documentation.

Conclusion

RAGFlow is an easy-to-use RAG framework. Although it does not provide a default ARM64 image, you can still build it from source to obtain a functional version of RAGFlow.