Update README.md
Browse files
README.md
CHANGED
|
@@ -9,7 +9,7 @@ inference: false
|
|
| 9 |
|
| 10 |
# Wizard-Vicuna-13B-GPTQ
|
| 11 |
|
| 12 |
-
This repo contains 4bit GPTQ format quantised models of [
|
| 13 |
|
| 14 |
It is the result of quantising to 4bit using [GPTQ-for-LLaMa](https://github.com/qwopqwop200/GPTQ-for-LLaMa).
|
| 15 |
|
|
@@ -17,7 +17,8 @@ It is the result of quantising to 4bit using [GPTQ-for-LLaMa](https://github.com
|
|
| 17 |
|
| 18 |
* [4bit GPTQ models for GPU inference](https://huggingface.co/TheBloke/wizard-vicuna-13B-GPTQ).
|
| 19 |
* [4bit and 5bit GGML models for CPU inference](https://huggingface.co/TheBloke/wizard-vicuna-13B-GGML).
|
| 20 |
-
|
|
|
|
| 21 |
## How to easily download and use this model in text-generation-webui
|
| 22 |
|
| 23 |
Open the text-generation-webui UI as normal.
|
|
@@ -53,7 +54,9 @@ It was created without the `--act-order` parameter. It may have slightly lower i
|
|
| 53 |
CUDA_VISIBLE_DEVICES=0 python3 llama.py wizard-vicuna-13B-HF c4 --wbits 4 --true-sequential --groupsize 128 --save_safetensors wizard-vicuna-13B-GPTQ-4bit.compat.no-act-order.safetensors
|
| 54 |
```
|
| 55 |
|
| 56 |
-
# Original
|
|
|
|
|
|
|
| 57 |
|
| 58 |
# WizardVicunaLM
|
| 59 |
### Wizard's dataset + ChatGPT's conversation extension + Vicuna's tuning method
|
|
|
|
| 9 |
|
| 10 |
# Wizard-Vicuna-13B-GPTQ
|
| 11 |
|
| 12 |
+
This repo contains 4bit GPTQ format quantised models of [junelee's wizard-vicuna 13B](https://huggingface.co/junelee/wizard-vicuna-13b).
|
| 13 |
|
| 14 |
It is the result of quantising to 4bit using [GPTQ-for-LLaMa](https://github.com/qwopqwop200/GPTQ-for-LLaMa).
|
| 15 |
|
|
|
|
| 17 |
|
| 18 |
* [4bit GPTQ models for GPU inference](https://huggingface.co/TheBloke/wizard-vicuna-13B-GPTQ).
|
| 19 |
* [4bit and 5bit GGML models for CPU inference](https://huggingface.co/TheBloke/wizard-vicuna-13B-GGML).
|
| 20 |
+
* [float16 HF format model for GPU inference](https://huggingface.co/TheBloke/wizard-vicuna-13B-HF).
|
| 21 |
+
|
| 22 |
## How to easily download and use this model in text-generation-webui
|
| 23 |
|
| 24 |
Open the text-generation-webui UI as normal.
|
|
|
|
| 54 |
CUDA_VISIBLE_DEVICES=0 python3 llama.py wizard-vicuna-13B-HF c4 --wbits 4 --true-sequential --groupsize 128 --save_safetensors wizard-vicuna-13B-GPTQ-4bit.compat.no-act-order.safetensors
|
| 55 |
```
|
| 56 |
|
| 57 |
+
# Original WizardVicuna-13B model card
|
| 58 |
+
|
| 59 |
+
Github page: https://github.com/melodysdreamj/WizardVicunaLM
|
| 60 |
|
| 61 |
# WizardVicunaLM
|
| 62 |
### Wizard's dataset + ChatGPT's conversation extension + Vicuna's tuning method
|