Update README.md
Browse files
README.md
CHANGED
|
@@ -14,7 +14,7 @@ base_model: zai-org/GLM-4.6V
|
|
| 14 |
|
| 15 |
### Quantization Details
|
| 16 |
|
| 17 |
-
- **Quantization Method:** AWQ
|
| 18 |
- **Bits:** 8
|
| 19 |
- **Group Size:** 32
|
| 20 |
- **Calibration Dataset:** [5CD-AI/LLaVA-CoT-o1-Instruct](https://huggingface.co/datasets/5CD-AI/LLaVA-CoT-o1-Instruct)
|
|
@@ -45,6 +45,10 @@ vllm serve cyankiwi/GLM-4.6V-AWQ-8bit
|
|
| 45 |
|
| 46 |
## Additional Information
|
| 47 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 48 |
### Changelog
|
| 49 |
|
| 50 |
- **v1.0.0** - Initial quantized release
|
|
|
|
| 14 |
|
| 15 |
### Quantization Details
|
| 16 |
|
| 17 |
+
- **Quantization Method:** cyankiwi AWQ v1.0
|
| 18 |
- **Bits:** 8
|
| 19 |
- **Group Size:** 32
|
| 20 |
- **Calibration Dataset:** [5CD-AI/LLaVA-CoT-o1-Instruct](https://huggingface.co/datasets/5CD-AI/LLaVA-CoT-o1-Instruct)
|
|
|
|
| 45 |
|
| 46 |
## Additional Information
|
| 47 |
|
| 48 |
+
### Known Issues
|
| 49 |
+
|
| 50 |
+
- `tensor-parallel-size > 2` requires `--enable-expert-parallel`
|
| 51 |
+
|
| 52 |
### Changelog
|
| 53 |
|
| 54 |
- **v1.0.0** - Initial quantized release
|