[hari2353]

Summary

This PR fixes the ARM64 (Apple Silicon) build and runtime of bitnet.cpp for the
official BitNet-b1.58-2B-4T i2_s model. Before this change, setup_env.py -q i2_s
failed to link on arm64 and, even after manual source fixes, run_inference.py
hung (infinite loop) or SIGSEGV'd.

Changes

1. Build wiring (vendored llama.cpp submodule)

The vendored ggml/src/CMakeLists.txt is a 10-line stub that never compiles
src/ggml-bitnet-mad.cpp, so quantize_i2_s / dequantize_row_i2_s were
undefined on arm64.

• 3rdparty/llama.cpp/ggml/src/CMakeLists.txt: target_sources(ggml-base PRIVATE ../../../../src/ggml-bitnet-mad.cpp) + include dirs.
• ggml-cpu/ggml-cpu.c: declare src1_cont in the I2_S fast path (was referenced
  undeclared at line ~1495).
• ggml-cpu/quants.c: guard dequantize_row_i2_s against reading past the packed
  data on the final partial block.
• ggml-cpu/ggml-cpu.c: fix activation pointer addressing i11 * nb11 (was
  i11 * nb11 / 4). Activations are i8 (1 byte), not 4-per-byte packed; the /4
  caused out-of-bounds reads and degenerate output. This was the root cause of
  the SIGSEGV.
• src/ggml-bitnet-mad.cpp: duplicate dequantize_row_i2_s (referenced by
  ggml.c's type-traits table but defined only in ggml-cpu/quants.c, unreachable
  from ggml-base); guard last-block reads.

Submodule commit: hari2353/llama.cpp@8d3a7d4 (branch bitnet-arm64-i2s-fix).

2. Runtime (run_inference.py)

• Add --single-turn so llama-cli exits after generating instead of entering the
  interactive loop (which spun forever / crashed on EOF stdin).

Verification (local, Apple M5 Pro, arm64)

• cmake --build build succeeds.
• run_inference.py -m ...ggml-model-i2_s.gguf -p "The capital of France is" -n 8
  now loads, generates, and exits cleanly (no hang, no SIGSEGV). Previously:
  SIGSEGV(11) / infinite loop.

Known limitation (not fixed by this PR)

Output is still degenerate (repeated @ tokens) for the test prompts. The matmul
kernel ggml_vec_dot_i2_i8_s (scalar fallback on ARM) and/or the activation
scale convention may need further attention from maintainers; the /4 fix above
removed the crash but the kernel's numerical result is still wrong. I'm happy to
iterate with guidance on the expected i2_s layout on ARM.

Test plan

• setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_s completes on arm64.
• run_inference.py runs to completion without crash/hang.
• Coherent text output (blocked on the kernel correctness issue above).

[hari2353]

@microsoft-github-policy-service agree [company=""]

[hari2353]

@microsoft-github-policy-service agree