sumink/enlow

Model: LLaMA (IFD Top 30% + Low Instruction Entropy)

🔍 Purpose

Fine-tune meta-llama/Llama-3.2-1B on instructions with low entropy among the high-IFD group.
These instructions tend to be short, simple, and repetitive.

📂 Dataset

• alpaca2000_entropy_low.csv
  • From alpaca2000.csv (IFD 상위 30%)
  • instruction entropy 하위 30% 추출 (약 180개)

⚙️ Training Config

• Model: meta-llama/Llama-3.2-1B
• Precision: bf16 or float32
• Epochs: 3
• Max length: 2048
• Output: output/llama_entropy_low

🧪 Goal

Measure whether simpler instructions lead to better model learning by reducing prompt complexity.