Unable to build the project on Raspberry Pi 4 #240

Activity

1. Benjamin-Wegener commented on Apr 28, 2025

   

   Benjamin Wegener (Benjamin-Wegener)

   on Apr 28, 2025

   Contributor

   More actions

   try
   cmake .. -DCMAKE_CXX_FLAGS="-O2"
   This avoids the crash by reducing optimization level and skipping problematic vectorization.
2. ping-ee commented on Apr 29, 2025

   

   ping-ee (ping-ee)

   on Apr 29, 2025

   Author

   More actions

   try cmake .. -DCMAKE_CXX_FLAGS="-O2" This avoids the crash by reducing optimization level and skipping problematic vectorization.

   Neither the compile flag -DCMAKE_CXX_FLAGS=-O2 nor -DCMAKE_CXX_FLAGS_RELEASE="-O2 -DNDEBUG" works for me.
3. ping-ee commented on Apr 29, 2025

   

   ping-ee (ping-ee)

   on Apr 29, 2025

   Last edited by ping-ee

   Author

   More actions

   I followed steps 5 and 6 mentioned on this page instead of running setup_env.py, and it works.

   👍React with 👍2Reacted by 이정한 and Chien Do
4. worldbiomusic commented on May 1, 2025

   

   이정한 (worldbiomusic)

   on May 1, 2025

   More actions

   I followed steps 5 and 6 mentioned on this page instead of running setup_env.py, and it works.

   It works. Thank you💖💖
5. Zodijackyl98 commented on Feb 8, 2026

   

   Mert Yasin Şafak (Zodijackyl98)

   on Feb 8, 2026

   More actions

   I followed everything I found on the internet and tried to solve the issue with the help of multiple AI agents but still I end up getting the same error. I don't think there is a hardware wall that I can't overcome since there are people ran BitNet on Pi 4
   like the one I have. So far I've tried these;

   • Using different compilers; Cling 18, 20 and Cmake.
   • Following official Github repo instructions and others I found online including the ones that run successfully on Pi 4 Here.
   • Editing ~/BitNet/3rdparty/llama.cpp/ggml/src/ggml-quants.c
   • Running cmake command with different flags such as

   cmake .. -DCMAKE_BUILD_TYPE=Release \
            -DCMAKE_C_COMPILER=/usr/bin/clang-18 \
            -DCMAKE_CXX_COMPILER=/usr/bin/clang++-18 \
            -DGGML_NATIVE=OFF \
            -DBITNET_ARM_TL1=OFF \
            -DCMAKE_C_FLAGS="-march=armv8-a" \
            -DCMAKE_CXX_FLAGS="-march=armv8-a"
   

   • Building the project with and without setup_env.py file.
     Here is the error;

   warning: not compiled with GPU offload support, --gpu-layers option will be ignored
   warning: see main README.md for information on enabling GPU BLAS support
   build: 3962 (1f86f058) with Debian clang version 18.1.8 (++20240731024826+3b5b5c1ec4a3-1~exp1~20240731144843.145) for aarch64-unknown-linux-gnu
   main: llama backend init
   Error occurred while running command: Command '['build/bin/llama-cli', '-m', 'models/BitNet-b1.58-2B-4T/ggml-model-i2_s.gguf', '-n', '128', '-t', '2', '-p', 'You are a helpful assistant', '-ngl', '0', '-c', '2048', '--temp', '0.8', '-b', '1', '-cnv']' died with <Signals.SIGILL: 4>.
   

   • AI focuses on this one line specifically -- Performing Test COMPILER_SUPPORTS_ARMV82_DOTPROD - Success since the CPU does not support dotprod and i8mm it behaves as if it supports.

   Here is the ouput after compiling;

    -- The C compiler identification is Clang 18.1.8
   
   -- The CXX compiler identification is Clang 18.1.8
   
   -- Detecting C compiler ABI info
   
   -- Detecting C compiler ABI info - done
   
   -- Check for working C compiler: /usr/bin/clang-18 - skipped
   
   -- Detecting C compile features
   
   -- Detecting C compile features - done
   
   -- Detecting CXX compiler ABI info
   
   -- Detecting CXX compiler ABI info - done
   
   -- Check for working CXX compiler: /usr/bin/clang++-18 - skipped
   
   -- Detecting CXX compile features
   
   -- Detecting CXX compile features - done
   
   -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
   
   -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
   
   -- Found Threads: TRUE  
   
   -- Found Git: /usr/bin/git (found version "2.39.5") 
   
   -- Could NOT find OpenMP_C (missing: OpenMP_C_FLAGS OpenMP_C_LIB_NAMES) 
   
   -- Could NOT find OpenMP_CXX (missing: OpenMP_CXX_FLAGS OpenMP_CXX_LIB_NAMES) 
   
   -- Could NOT find OpenMP (missing: OpenMP_C_FOUND OpenMP_CXX_FOUND) 
   
   CMake Warning at 3rdparty/llama.cpp/ggml/src/CMakeLists.txt:170 (message):
   
     OpenMP not found
   
   
   
   -- Using llamafile
   
   -- ccache found, compilation results will be cached. Disable with GGML_CCACHE=OFF.
   
   -- CMAKE_SYSTEM_PROCESSOR: aarch64
   
   -- ARM detected
   
   -- Performing Test COMPILER_SUPPORTS_FP16_FORMAT_I3E
   
   -- Performing Test COMPILER_SUPPORTS_FP16_FORMAT_I3E - Failed
   
   -- Performing Test GGML_COMPILER_SUPPORT_DOTPROD
   
   -- Performing Test GGML_COMPILER_SUPPORT_DOTPROD - Failed
   
   -- Performing Test COMPILER_SUPPORTS_ARMV82_DOTPROD
   
   -- Performing Test COMPILER_SUPPORTS_ARMV82_DOTPROD - Success
   
   -- Configuring done
   
   -- Generating done
   
   -- Build files have been written to: /home/user/BitNet/build
   
6. lepus2589 commented on Feb 22, 2026

   

   Tim Kaune (lepus2589)

   on Feb 22, 2026

   Last edited by lepus2589

   More actions

   I encountered the SIGILL signal (illegal instruction), too. I tried to build the latest main (8fd3412) on the latest Raspberry Pi OS from December 2025 based on Debian 13 (Trixie).

   This gives you Clang/LLVM v19 and Python v3.13. Which unfortunately meant, that I had to build Python v3.12 from source, as the PyTorch version required by the BitNet project only works for Python<=3.12.

   But after I solved that problem, I followed the steps from Bijan Bowen's (Bijan Bowen (@OminousIndustries)) BitNet b1.58 on Raspberry Pi 4B blog post. After setting up the Python virtual environment, I did:

   1. Generating the header and config files with the command from setup_env.py#L180:

      $ python utils/codegen_tl1.py \
        --model bitnet_b1_58-3B \
        --BM 160,320,320 \
        --BK 64,128,64 \
        --bm 32,64,32

   2. Building with CMake:

      $ export CC=clang-19
      $ export CXX=clang++-19
      $ cmake -B ./build -DCMAKE_BUILD_TYPE=Release -DBITNET_ARM_TL1=OFF .
      $ cmake --build ./build --verbose -j4

   Running python run_inference.py ... caused the SIGILL error.

   Because some SO answers pointed to a problem with the armv8-a +lse extension being assumed but unavailable on the Pi 4B, I even configured the complete arch and cpu info in CMake like this:

   $ cmake -B ./build \ 
     -DCMAKE_BUILD_TYPE=Release \
     -DBITNET_ARM_TL1=OFF \
     -DCMAKE_C_FLAGS="-march=armv8-a+nolse -mcpu=cortex-a72 -mtune=cortex-a72 -mno-outline-atomics" \
     -DCMAKE_CXX_FLAGS="-march=armv8-a+nolse -mcpu=cortex-a72 -mtune=cortex-a72 -mno-outline-atomics" \
     .

   But that didn't help.

   Next, I tried to checkout a revision from the approximate time of Bijan Bowen's blog post. I decided for c17d1c5, as it already had the inference server and the model architecture name fix in it. I deleted the ./build directory, regenerated the header files and rebuilt with CMake and, voilà, it worked!

   I decided to do a git bisect and it turns out, that the commit introducing this regression is c17d1c5. The last commit working on the RPi 4B is 404980e.

   So, for anyone trying this in 2026, do this after cloning the repository:

   $ git switch --detach 404980eecae38affa4871c3e419eae3f44536a95
   $ git submodule update --recursive

   Mert Yasin Şafak (@Zodijackyl98) Your AIs might be onto something there. The GGML_COMPILER_SUPPORT_DOTPROD try_compile() done by the 3rdparty/llama.cpp/ggml/src/CMakeLists.txt first tests, if the dotprod extension is supported by the target system. If you are on a system with an armv8.2-a+dotprod arch, this should succeed and simply add the respective pre-processor define. If it fails (as it does on RPi 4B), we follow up with the COMPILER_SUPPORTS_ARMV82_DOTPROD test to see, if the compiler supports the -march=armv8.2-a+dotprod flag (so basically, if you could cross-compile for such a system). If that succeeds, we explicitly set the -march=armv8.2-a+dotprod flag (and thus are now basically cross-compiling for such a CPU). If you are on a armv8-a (like the RPi 4B) and your compiler supports cross-compiling for armv8.2-a+dotprod (which clang-19 apparently does), you are now cross-compiling for such a system instead of compiling for your host system. Coincidentally, this try_compile() block was introduced in the "bad" commit.

   Now, that I was explicitly looking for it, I actually saw the -march=armv8.2-a+dotprod given to the compiler. No wonder, it produces invalid instructions.

   I think, the complete COMPILER_SUPPORTS_ARMV82_DOTPROD test needs to go. It only tests the cross-compile capability of the compiler, which doesn't help at all determining the capabilities of the target system:

   diff --git a/ggml/src/CMakeLists.txt b/ggml/src/CMakeLists.txt
   index 9006bdb7..197c4f4a 100644
   --- a/ggml/src/CMakeLists.txt
   +++ b/ggml/src/CMakeLists.txt
   @@ -1155,12 +1155,12 @@ if (CMAKE_OSX_ARCHITECTURES      STREQUAL "arm64" OR
            check_cxx_source_compiles("#include <arm_neon.h>\nint main(){ int8x16_t a{}, b{}; int32x4_t s = vdupq_n_s32(0); s = vdotq_s32(s,a,b); (void)s; return 0; }" GGML_COMPILER_SUPPORT_DOTPROD)
            if (GGML_COMPILER_SUPPORT_DOTPROD)
                add_compile_definitions(__ARM_FEATURE_DOTPROD)
   -        else()
   -            check_cxx_compiler_flag("-march=armv8.2-a+dotprod" COMPILER_SUPPORTS_ARMV82_DOTPROD)
   -            if (COMPILER_SUPPORTS_ARMV82_DOTPROD)
   -                list(APPEND ARCH_FLAGS -march=armv8.2-a+dotprod)
   -                add_compile_definitions(__ARM_FEATURE_DOTPROD)
   -            endif()
            endif()
        endif()
    elseif (CMAKE_OSX_ARCHITECTURES STREQUAL "x86_64" OR CMAKE_GENERATOR_PLATFORM_LWR MATCHES "^(x86_64|i686|amd64|x64|win32)$" OR

   I tried this on main and it compiles and runs, but now it produces gibberish when running inference. This looks like what is described in #411.

   Until these bugs are fixed, the "good" commit above is the only thing that works.

   👍React with 👍1Reacted by Gun Deniz Akkoc