---
language:
- en
license: apache-2.0
library_name: transformers
tags:
- qwen
- qwen3
- lora
- peft
- fine-tuning
- medical
- causal-lm
- 4bit
datasets:
- Rabe3/QA_Synthetic_Medical_data
metrics:
- perplexity
base_model: Qwen/Qwen3-0.6B
model-index:
- name: Qwen3-0.6B-LoRA-Medical
  results:
  - task:
      type: text-generation
    dataset:
      name: Rabe3/QA_Synthetic_Medical_data
      type: Rabe3/QA_Synthetic_Medical_data
    metrics:
    - type: loss
      value: 0.2094
      name: Training Loss
    - type: perplexity
      value: 1.23
      name: Perplexity
---

# Qwen3-0.6B LoRA — Medical QA Fine-Tuned Model

This model is a **LoRA-fine-tuned version of [Qwen/Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B)** using the dataset [Rabe3/QA_Synthetic_Medical_data](https://huggingface.co/datasets/Rabe3/QA_Synthetic_Medical_data).  
It is optimized for concise, context-aware question-answering and instruction following in medical domains.

---

## Model Details

- **Base model:** Qwen/Qwen3-0.6B  
- **Fine-tuned by:** [MyungHwan Hong](https://huggingface.co/MightyOctopus)  
- **Framework:** Transformers + PEFT (LoRA)  
- **Quantization:** 4-bit (nf4, bitsandbytes)  
- **Training type:** Causal LM (instruction fine-tuning)  
- **Language:** English (medical focus)  
- **License:** Apache-2.0  

---

## Training Details

| Hyperparameter | Value |
|----------------|--------|
| Epochs | 50 |
| Learning rate | 1e-3 |
| Batch size | 4 |
| Gradient accumulation | 4 |
| Precision | fp16 |
| Optimizer | AdamW |
| Quantization | 4-bit nf4 |
| Loss | 0.2094 |
| Perplexity | 1.23 |

Dataset used: **Rabe3/QA_Synthetic_Medical_data**

---

## Intended Use

**Direct use:**  
- Educational or research models for medical QA  
- Domain-specific chatbot prototypes  
- Language understanding in synthetic medical text

**Not suitable for:**  
- Clinical or diagnostic purposes  
- High-stakes or real-world medical use

---

## Limitations & Bias

- This model was trained on synthetic medical text, which may not reflect real clinical data.  
- May generate plausible-sounding but incorrect information (“hallucination”).  
- Should never replace qualified medical judgment.

---

## How to Use

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "MightyOctopus/qwen3-0.6B-lora-medical"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

prompt = "What are common symptoms of Type 2 diabetes?"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))