IQ4_KSS
Adding findings for IQ4_KSS
Xeon 8480+ QYFS 512 RAM 5600mhz 2x 3090
numactl -N 0 -m 0 ../build/bin/llama-server
--model "/mnt/ExtraStorage/Models/Kimi-K2-Thinking-smol-IQ4_KSS-00001-of-00011.gguf"
--alias ubergarm\KimiThink-IQ4_KSS
-mla 3 -amb 2048 -b 4096 -ub 4096
-c 32768 -ctk q8_0
-ngl 99
-ot exps=CPU
--parallel 1
--threads 88
--host 127.0.0.1
--port 8080
--numa numactl
--split-mode layer --tensor-split 1,1
--mirostat 2 --mirostat-ent 3
--mirostat-lr 0.1
--special
Adding Butterfly for comparison to IQ3_K (Seems a little better at iq4)
Prompt for the butterfly?
Prompt for the butterfly?
simple prompt I used to make sure it looks something like a butterfly to make sure settings are right before I judge a model " create a beautiful, symmetrical butterfly in SVG"
Yeah the iq4_kss is looking pretty good on the perplexity graph.