Zaynoid/Qwen3-VL-30B-A3B

Qwen3-VL-30B Fine-tuned on Alpaca Dataset

This model is a fine-tuned version of Qwen/Qwen3-VL-30B-A3B-Instruct on the Zaynoid/pathrep-processed-alpaca dataset.

Training Details

• Base Model: Qwen3-VL-30B-A3B-Instruct
• Fine-tuning Method: LoRA (merged)
• Dataset: pathrep-processed-alpaca
• Training Epochs: 3
• Final Training Loss: 0.8265

Usage

import torch
from transformers import AutoModelForVision2Seq, AutoProcessor

model = AutoModelForVision2Seq.from_pretrained(
    "Zaynoid/qwen3vl-30b-alpaca-merged",
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True
)

processor = AutoProcessor.from_pretrained(
    "Zaynoid/qwen3vl-30b-alpaca-merged",
    trust_remote_code=True
)

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Your question here"}
]

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=text, return_tensors="pt").to(model.device)

with torch.no_grad():
    output_ids = model.generate(**inputs, max_new_tokens=512)

response = processor.decode(output_ids[0], skip_special_tokens=True)
print(response)

Training Configuration

• LoRA r=64, alpha=128
• Learning rate: 2e-4
• Batch size: 1 (per device)
• Gradient accumulation: 8 steps
• Optimizer: AdamW with cosine LR schedule
• Weight decay: 0.01
• Max gradient norm: 1.0