mradermacher

/

Qwen3-VL-8B-Instruct-abliterated-v2-i1-GGUF

like 1

Here a quick summery to hopefully clarify things:

• Base model: https://huggingface.co/prithivMLmods/Qwen3-VL-8B-Instruct-abliterated-v2
• Static quants: https://huggingface.co/mradermacher/Qwen3-VL-8B-Instruct-abliterated-v2-GGUF
• Weighted/imatrix quants: https://huggingface.co/mradermacher/Qwen3-VL-8B-Instruct-abliterated-v2-i1-GGUF
• Our convenient download page: https://hf.tst.eu/model#Qwen3-VL-8B-Instruct-abliterated-v2-GGUF

If unsure just go for weighted/imatrix quants as they are offer superior quality at the same model size and hardware requirements. Static quants are created by just quantizing certain tensors to certain bits per wights while for weighted/imatrix quants we measure for what parts of the model precision is more important by computing an importance matrix and then store in high precision what is important and in low precision what matter less resulting in an overall better model at all use cases we tested. Ideally you would check our convenient download page and take a look at the quality column. In there you can select a quality metric to filter which will help you to judge yourself which quants best fits your use-case.

i1 refere to what !?

i1 refere to what !?

That the importance matrix was computed using measurements made using our i1 dataset. The first half of our i1 dataset contains bartowski's imatrix dataset while the second half contains our own proprietary high-quality data covering all the use cases bartowski missed in his imatrix dataset while doubling the amount of measurements resulting in a better importance matrix. The plan was that should we ever majorly overhaul our imatrix dataset we start naming them i2 but everyone started using the i1 search term inside the HuggingFace search to find our imatrix quants, so this is unlikely to ever happen. In case you wonder about why we historically choose this specific name let's quote our Q&A from https://huggingface.co/mradermacher/model_requests:

What does the "-i1" mean in "-i1-GGUF"?
"mradermacher imatrix type 1"
Originally, I had the idea of using an iterational method of imatrix generation, and wanted to see how well it fares. That is, create an imatrix from a bad quant (e.g. static Q2_K), then use the new model to generate a possibly better imatrix. It never happened, but I think sticking to something, even if slightly wrong, is better changing it. If I make considerable changes to how I create imatrix data I will probably bump it to -i2 and so on.
since there is some subjectivity/choice in imatrix training data, this also distinguishes it from quants by other people who made different choices.