---
license: apache-2.0
base_model: Qwen/Qwen3-VL-30B-A3B-Instruct
tags:
- vision
- qwen3-vl
- fine-tuned
- alpaca
datasets:
- Zaynoid/pathrep-processed-alpaca
language:
- en
---

# Qwen3-VL-30B Fine-tuned on Alpaca Dataset

This model is a fine-tuned version of [Qwen/Qwen3-VL-30B-A3B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-30B-A3B-Instruct) on the [Zaynoid/pathrep-processed-alpaca](https://huggingface.co/datasets/Zaynoid/pathrep-processed-alpaca) dataset.

## Training Details

- **Base Model:** Qwen3-VL-30B-A3B-Instruct
- **Fine-tuning Method:** LoRA (merged)
- **Dataset:** pathrep-processed-alpaca
- **Training Epochs:** 3
- **Final Training Loss:** 0.8265

## Usage

```python
import torch
from transformers import AutoModelForVision2Seq, AutoProcessor

model = AutoModelForVision2Seq.from_pretrained(
    "Zaynoid/qwen3vl-30b-alpaca-merged",
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True
)

processor = AutoProcessor.from_pretrained(
    "Zaynoid/qwen3vl-30b-alpaca-merged",
    trust_remote_code=True
)

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Your question here"}
]

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=text, return_tensors="pt").to(model.device)

with torch.no_grad():
    output_ids = model.generate(**inputs, max_new_tokens=512)

response = processor.decode(output_ids[0], skip_special_tokens=True)
print(response)
```

## Training Configuration

- LoRA r=64, alpha=128
- Learning rate: 2e-4
- Batch size: 1 (per device)
- Gradient accumulation: 8 steps
- Optimizer: AdamW with cosine LR schedule
- Weight decay: 0.01
- Max gradient norm: 1.0