modelcard.md

metadata

license: apache-2.0
tags:
  - music
  - text-generation
  - instruction-tuning
  - lora
  - preview
  - untrained
  - qwen3.5
  - touchgrass
datasets:
  - synthetic
language:
  - en
library_name: transformers
pipeline_tag: text-generation

TouchGrass-3B 🎵

Status: PREVIEW - UNTRAINED MODEL

This is a preview repository for TouchGrass-3B, a lightweight music AI assistant fine-tuned from Qwen3.-3B-Instruct. This model has NOT been trained yet - it contains randomly initialized LoRA adapters and is not ready for inference.

⚠️ Important Notice

• Model is UNTRAINED: The LoRA adapters are randomly initialized. Performance will be no better than the base Qwen3.5-3B-Instruct model.
• For demonstration purposes only: This repository contains the complete codebase and configuration for training the model.
• Expected performance after training: 94-95% accuracy on music-specific tasks (based on architecture design and synthetic data pipeline).

🎯 Model Overview

TouchGrass is a specialized music AI assistant built by fine-tuning Qwen3.5 models with:

• Music Tokenizer Extension: 21+ music-specific tokens (guitar, piano, drums, vocals, theory, DJ, tablature, chords, etc.)
• Five Specialized Modules:
  • 🎸 Tab & Chord Generation (guitar tabs, chord diagrams)
  • 🎹 Music Theory Engine (scales, intervals, progressions)
  • 👂 Ear Training (interval ID, solfege exercises)
  • 😌 EQ Adapter (frustration detection, emotional adaptation)
  • ✍️ Song Writing Assistant (progressions, lyrics, hooks)
• LoRA Fine-Tuning: Efficient parameter-efficient fine-tuning
• Multi-Task Learning: Weighted losses (LM: 1.0, EQ: 0.1, Music: 0.05)

📊 Model Details

Property	Value
Base Model	Qwen/Qwen3.5-3B-Instruct
Model Size	~3.5B parameters (with LoRA)
Vocab Size	32,000 (Qwen3.5 + music tokens)
Max Sequence Length	4,096 tokens
LoRA Rank	16 (configurable)
Training Data	Synthetic music QA (10 categories, 80+ templates)
Training Steps	50,000 (planned)
Batch Size	8-16 (depending on GPU)
Learning Rate	2e-4 (with warmup)

🏗️ Architecture

The model extends Qwen3.5 with:

1. Custom tokenizer with music domain tokens
2. Five LoRA-adapted modules inserted at transformer layers
3. Multi-task heads for music-specific predictions
4. Emotional intelligence via EQ adapter

🚀 Usage (After Training)

HuggingFace Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
from TouchGrass.configuration_touchgrass import TouchGrassConfig
from TouchGrass.tokenization_touchgrass import TouchGrassTokenizer

# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained("your-username/TouchGrass-3B")
tokenizer = TouchGrassTokenizer.from_pretrained("your-username/TouchGrass-3B")

# Generate with instrument context
prompt = "[GUITAR][BEGINNER] How do I play an F major chord?"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0]))

Ollama (After Training)

# Create Modelfile (provided in repository)
ollama create touchgrass-3b -f ollama_3b_modelfile

# Run inference
ollama run touchgrass-3b "How do I build a chord progression in C major?"

📁 Repository Structure

This repository contains all necessary files for training:

touchgrass-3b/
├── configuration_touchgrass.py   # HuggingFace config class
├── tokenization_touchgrass.py    # HuggingFace tokenizer wrapper
├── train.py                      # Main training script
├── configs/
│   ├── touchgrass_3b_config.py  # Model architecture config
│   ├── touchgrass_7b_config.py  # 7B config (for reference)
│   └── training_config.py       # Training hyperparameters
├── tokenizer/
│   └── music_token_extension.py # Music token definitions
├── models/                      # Five specialized modules
│   ├── tab_chord_module.py
│   ├── music_theory_module.py
│   ├── ear_training_module.py
│   ├── eq_adapter.py
│   └── songwriting_module.py
├── data/                        # Data pipeline
│   ├── music_qa_generator.py
│   ├── chat_formatter.py
│   └── dataset_loader.py
├── training/
│   ├── losses.py
│   ├── trainer.py
│   └── train.py
├── inference/
│   └── inference.py
├── benchmarks/
│   ├── evaluate_music_modules.py
│   └── evaluate_inference.py
├── tests/                       # Comprehensive test suite
├── ollama_3b_modelfile         # Ollama configuration
├── README.md                   # Full documentation
└── PREVIEW_README.md           # This preview notice

🧪 Testing

Run the test suite:

cd touchgrass-3b
python -m pytest tests/ -v

📚 Documentation

See README.md for complete documentation including:

• Installation instructions
• Training guide
• Inference examples
• Module specifications
• Data generation details
• Troubleshooting

⚙️ Training (When Resources Available)

1. Generate synthetic data:

python -c "from data.music_qa_generator import MusicQAGenerator; MusicQAGenerator().generate_dataset(num_samples=10000, output_path='data/music_qa.jsonl')"

2. Start training:

python train.py --config configs/touchgrass_3b_config.py --data data/music_qa.jsonl --output_dir ./checkpoints

3. Convert to HuggingFace format:

python -c "from configuration_touchgrass import TouchGrassConfig; from tokenization_touchgrass import TouchGrassTokenizer; config = TouchGrassConfig.from_pretrained('./checkpoints'); tokenizer = TouchGrassTokenizer.from_pretrained('./checkpoints'); config.save_pretrained('./model'); tokenizer.save_pretrained('./model')"

4. Push to HuggingFace:

huggingface-cli login
huggingface-cli upload your-username/TouchGrass-3B ./model --repo-type model

🤝 Contributing

This is a preview. Contributions welcome for:

• Improving synthetic data quality
• Adding more music categories
• Optimizing training efficiency
• Extending to more instruments

📄 License

Apache 2.0

🙏 Acknowledgments

• Built upon Qwen3.5 by Alibaba Cloud
• Inspired by the need for accessible music education AI
• Special thanks to the open-source music technology community

---

⚠️ REMINDER: This is an UNTRAINED PREVIEW model. Do not use for production inference without completing the training process.