These are quantizations of the model Huihui-Ling-mini-2.0-abliterated.

These quantizations were created using an imatrix merged from combined_all_large and harmful.txt to leverage the abliterated nature of the model.

GGUF

Model size

16B params

Architecture

bailingmoe2

Hardware compatibility

3-bit

4-bit

5-bit

6-bit

8-bit

Model tree for noctrex/Huihui-Ling-mini-2.0-abliterated-i1-GGUF

Base model

Finetuned

Quantized

Quantized

(8)

this model