Published BitNet embedding GGUF files declare MEAN pooling; model card says last-token #640

Summary

The two published embedding GGUF files declare pooling_type = 1 (MEAN). The model card and docs/bitnet-embeddings-i2s-guide.md both say the models use last-token (EOS) pooling. llama-embedding follows the file, so the documented command produces mean-pooled vectors. On a retrieval test of mine, passing --pooling last roughly doubled rank-1 accuracy on one set and raised it less on the other.

Files

• microsoft/bitnet-embedding-0.6b, bitnet-embeddings-0.6b-bf16-i2_s.gguf, revision 16176d1
• microsoft/bitnet-embedding-270m, bitnet-embeddings-270m-bf16-i2_s.gguf, revision 5f1c2fd

What the files say

Loading each with llama-embedding -v:

llama_model_loader: - kv  15:   qwen3.pooling_type u32 = 1      (0.6b)
llama_model_loader: - kv  15:  gemma3.pooling_type u32 = 1      (270m)
print_info: pooling type = 1

In include/llama.h, LLAMA_POOLING_TYPE_MEAN = 1 and LLAMA_POOLING_TYPE_LAST = 3.

What the documentation says

Model card and guide: "Pooling Strategy: Last-token (EOS) pooling followed by L2 normalization."

Likely cause

utils/convert-bitnet-embedding-to-gguf.py falls back to MEAN when it finds no sentence-transformers pooling config:

if pooling_type is None:
    # Default to MEAN pooling for embedding models
    logger.info("  No pooling config found, defaulting to MEAN pooling")
    pooling_type = gguf.PoolingType.MEAN

Effect

Heading-to-passage retrieval on English regulatory and legal text, 300 queries against 600 passages per set, same build, 6 threads. Share of queries with the right passage ranked first, across three query forms (no prefix, query: , and Instruct: ...\nQuery: ):

With the file default, the instruction form was the worst of the three for both models. With --pooling last it was the best for the 0.6b on both sets, and still the worst for the 270m.

The test data is private, so I cannot share it. The metadata above is reproducible from the published files alone.

Environment

• microsoft/BitNet 0b341e5, submodule isHuangXin/llama.cpp at 390c307 (branch release-bitnet-embedding-0.6b-270m)
• Built per the guide's "Option 2: Using CMake directly", clang 18.1.3, GGML_NATIVE=ON
• x86-64 Linux, AMD EPYC (AVX2, AVX-512)

Suggested fix

• Re-publish both GGUF files with pooling_type = LAST, or have the converter set LAST for these two architectures.
• Until then, add --pooling last to the llama-embedding examples in the model card and the guide.

One question

The FAQ says queries need a one-sentence instruction, but I could not find the exact template. Is it Instruct: {task}\nQuery: {query}? Stating it in the card would help.