---
license: apache-2.0
tags:
  - gptq
  - qwen
  - text-generation
  - causal-lm
  - transformers
---

# Qwen2.5-3B GPTQ

This is the **Qwen2.5-3B** model quantized using **GPTQ** for efficient inference with `AutoGPTQ`.

### 🔍 Model Details:
- Base: [Qwen2.5-3B](https://huggingface.co/Qwen/Qwen2.5-3B)
- Quantization: GPTQ
- Format: `safetensors` (safe and fast loading)
- Purpose: Fast inference for chat and code tasks

---

## 🧪 Example Usage (Transformers)

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "STiFLeR7/Qwen2.5-3B-GPTQ"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto", trust_remote_code=True)

prompt = "Q: Who is Alan Turing?\nA:"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))