MIXED_V6 - Combined Clinical + Standard Reasoning Model

Overview

Fine-tuned Llama 3.1 8B on a mixed dataset combining Clinical Reasoning (CRD) and Chain-of-Thought (CoT) data.

Training Details

• Base Model: meta-llama/Meta-Llama-3.1-8B-Instruct
• Training Framework: Unsloth
• Dataset: Mixed CRD + CoT (mixed_crd_cot_10k_unsloth.jsonl)
• Examples: 10000
• Training Time: 0.27 hours
• Final Loss: 1.4995

Hyperparameters (Balanced)

• Learning Rate: 1.5e-05
• Max Steps: 400
• LoRA Rank: 8
• LoRA Alpha: 16
• LoRA Dropout: 0.05
• Warmup: 10%
• Max Sequence Length: 2048
• Effective Batch Size: 8

Dataset Composition

• ~50% Clinical Reasoning Data (medical)
• ~50% Chain-of-Thought Reasoning (general)

Part of Experiment

• kinzakhan1/CRD_V6 - Clinical reasoning only
• kinzakhan1/SRD_V6 - Standard reasoning only
• kinzakhan1/MIXED_V6 - Mixed dataset (this model)