MightyOctopus/qwen3-0.6B-lora-medical

Qwen3-0.6B LoRA — Medical QA Fine-Tuned Model

This model is a LoRA-fine-tuned version of Qwen/Qwen3-0.6B using the dataset Rabe3/QA_Synthetic_Medical_data.
It is optimized for concise, context-aware question-answering and instruction following in medical domains.

---

Model Details

• Base model: Qwen/Qwen3-0.6B
• Fine-tuned by: MyungHwan Hong
• Framework: Transformers + PEFT (LoRA)
• Quantization: 4-bit (nf4, bitsandbytes)
• Training type: Causal LM (instruction fine-tuning)
• Language: English (medical focus)
• License: Apache-2.0

---

Training Details

Hyperparameter	Value
Epochs	50
Learning rate	1e-3
Batch size	4
Gradient accumulation	4
Precision	fp16
Optimizer	AdamW
Quantization	4-bit nf4
Loss	0.2094
Perplexity	1.23

Dataset used: Rabe3/QA_Synthetic_Medical_data

---

Intended Use

Direct use:

• Educational or research models for medical QA
• Domain-specific chatbot prototypes
• Language understanding in synthetic medical text

Not suitable for:

• Clinical or diagnostic purposes
• High-stakes or real-world medical use

---

Limitations & Bias

• This model was trained on synthetic medical text, which may not reflect real clinical data.
• May generate plausible-sounding but incorrect information (“hallucination”).
• Should never replace qualified medical judgment.

---

How to Use

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "MightyOctopus/qwen3-0.6B-lora-medical"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

prompt = "What are common symptoms of Type 2 diabetes?"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))