Llama.cpp
-
SQ4 Is Live: Whitepaper + llama.cpp PR
We published the SQ4 whitepaper and opened a PR to bring percentile-calibrated 4-bit quantization to llama.cpp.
We published the SQ4 whitepaper and opened a PR to bring percentile-calibrated 4-bit quantization to llama.cpp.