laya-gguf

GGUF conversion of convaiinnovations/laya (Apache-2.0), f16, via llama.cpp converthftogguf.py. laya-F16.gguf (and the quants) hold the ModernBERT backbone only, and load in llama.cpp as modern-bert (use --embeddings --pooling none). The decision head and config are in the sibling file laya-head.safetensors (like an mmproj file); read it with layahead.py (loadhead(path) returns config + head weights under the original PyTorch names). The head itself must ru...

# 1. serve the backbone (per-token hidden states; -ub must cover your longest input)
llama-server -m laya-F16.gguf --embeddings --pooling none -c 2048 -ub 2048 -b 2048 --port 8080

# 2. python deps for the decision head + tokenizer
pip install laya requests torch safetensors