Hugging Face's logo

RichardErkhov
/
nldemo_-_Llama-3-8B-Story-Summarization-QLoRA-gguf

GGUF
Model card Files
xet
Community
YAML Metadata Warning: empty or missing yaml metadata in repo card (https://huggingface.co/docs/hub/model-cards#model-card-metadata)

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Llama-3-8B-Story-Summarization-QLoRA - GGUF

Name Quant method Size
Llama-3-8B-Story-Summarization-QLoRA.Q2_K.gguf Q2_K 2.96GB
Llama-3-8B-Story-Summarization-QLoRA.IQ3_XS.gguf IQ3_XS 3.28GB
Llama-3-8B-Story-Summarization-QLoRA.IQ3_S.gguf IQ3_S 3.43GB
Llama-3-8B-Story-Summarization-QLoRA.Q3_K_S.gguf Q3_K_S 3.41GB
Llama-3-8B-Story-Summarization-QLoRA.IQ3_M.gguf IQ3_M 3.52GB
Llama-3-8B-Story-Summarization-QLoRA.Q3_K.gguf Q3_K 3.74GB
Llama-3-8B-Story-Summarization-QLoRA.Q3_K_M.gguf Q3_K_M 3.74GB
Llama-3-8B-Story-Summarization-QLoRA.Q3_K_L.gguf Q3_K_L 4.03GB
Llama-3-8B-Story-Summarization-QLoRA.IQ4_XS.gguf IQ4_XS 4.18GB
Llama-3-8B-Story-Summarization-QLoRA.Q4_0.gguf Q4_0 4.34GB
Llama-3-8B-Story-Summarization-QLoRA.IQ4_NL.gguf IQ4_NL 4.38GB
Llama-3-8B-Story-Summarization-QLoRA.Q4_K_S.gguf Q4_K_S 4.37GB
Llama-3-8B-Story-Summarization-QLoRA.Q4_K.gguf Q4_K 4.58GB
Llama-3-8B-Story-Summarization-QLoRA.Q4_K_M.gguf Q4_K_M 4.58GB
Llama-3-8B-Story-Summarization-QLoRA.Q4_1.gguf Q4_1 4.78GB
Llama-3-8B-Story-Summarization-QLoRA.Q5_0.gguf Q5_0 5.21GB
Llama-3-8B-Story-Summarization-QLoRA.Q5_K_S.gguf Q5_K_S 5.21GB
Llama-3-8B-Story-Summarization-QLoRA.Q5_K.gguf Q5_K 5.34GB
Llama-3-8B-Story-Summarization-QLoRA.Q5_K_M.gguf Q5_K_M 5.34GB
Llama-3-8B-Story-Summarization-QLoRA.Q5_1.gguf Q5_1 5.65GB
Llama-3-8B-Story-Summarization-QLoRA.Q6_K.gguf Q6_K 6.14GB
Llama-3-8B-Story-Summarization-QLoRA.Q8_0.gguf Q8_0 7.95GB

Original model description:

language: - en license: apache-2.0 tags: - text-generation-inference - transformers - unsloth - llama - trl - sft base_model: unsloth/llama-3-8b-bnb-4bit

Uploaded model

  • Developed by: nldemo
  • License: apache-2.0
  • Finetuned from model : unsloth/llama-3-8b-bnb-4bit

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.

Downloads last month
297
GGUF
Model size
8B params
Architecture
llama
Hardware compatibility
Log In to view the estimation

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support