---
quantized_by: quantflex
pipeline_tag: text-generation
license: mit
base_model_relation: quantized
base_model: XiaomiMiMo/MiMo-7B-RL
language:
- zh
- en
---

**---- NOTE -----**

I have deleted the mtp layers in order to make it work with llama.cpp. Quality might be degraded.

A proper implementation would be better, but this will work until that is implemented.

For more information feel free to open a discussion here.

Here is a tutorial on how I made these quants without mtp:

https://huggingface.co/XiaomiMiMo/MiMo-7B-RL/discussions/5

**---- NOTE -----**

GGUF Quants for: [XiaomiMiMo/MiMo-7B-RL](https://huggingface.co/XiaomiMiMo/MiMo-7B-RL) 

Model by: [XiaomiMiMo](https://huggingface.co/XiaomiMiMo)

Quants by: [quantflex](https://huggingface.co/quantflex)

Run with [llama.cpp](https://github.com/ggerganov/llama.cpp):

```./llama-cli -m MiMo-7B-RL-nomtp-Q5_K_M.gguf -cnv```