Hugging Face's logo

marcosremar2
/
ensemble-tts-annotation

Text-to-Speech
Portuguese
tts
orpheus
ensemble
emotion-recognition
speech-emotion-recognition
annotation
portuguese
brazilian-portuguese
Model card Files
xet
Community
ensemble-tts-annotation
File size: 2,773 Bytes
3c495b0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
d669352
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
 
# 🚀 Real-Time Progress - SkyPilot Fine-tuning

**Status**: ⏳ IN PROGRESS
**Started**: 2025-12-02 13:00 UTC
**Cluster**: sky-33ba-marcos

---

## 📊 Current Job: Fine-tuning

### Machine Provisioned:
```
Provider: Vast.ai (Czechia, CZ, EU)
Instance: A100 SXM4
vCPUs: 32 cores
RAM: 64GB
GPU: A100 (1x)
Cost: $0.00/hr ✨ FREE!
```

### What's Running:
1. ✅ Machine provisioned
2. ⏳ Installing dependencies (torch, transformers, librosa)
3. ⏳ Cloning repository
4. ⏳ Creating synthetic data (50 samples/emotion)
5. ⏳ Preparing dataset
6. ⏳ Fine-tuning emotion2vec (10 epochs)
7. ⏳ Testing model

### Estimated Time:
- Setup: ~5min
- Data generation: ~1min
- Fine-tuning: ~20-30min
- Testing: ~2min
- **Total**: ~30-40min

### Expected Output:
```
✅ Fine-tuning complete!
Model saved to: models/emotion/emotion2vec_finetuned_synthetic/
```

---

## 📝 How to Monitor

### Check logs in real-time:
```bash
sky logs sky-33ba-marcos -f
```

### Check status:
```bash
sky status
```

### SSH to machine (while running):
```bash
sky ssh sky-33ba-marcos
# Inside:
cd ensemble-tts-annotation
watch -n 1 nvidia-smi  # Monitor GPU usage
```

---

## 💰 Cost Tracking

| Item | Cost |
|------|------|
| Validation test | $0.00 |
| Fine-tuning (current) | $0.00 (Vast.ai spot) |
| **Total so far** | **$0.00** ✨ |

---

## 🎯 After This Completes

### Next Steps:
1. **Download model**:
   ```bash
   sky scp sky-33ba-marcos:~/ensemble-tts-annotation/models/emotion/finetuned/ ./models/
   ```

2. **Test locally**:
   ```python
   from ensemble_tts import EnsembleAnnotator

   annotator = EnsembleAnnotator(mode='balanced', device='cuda')
   result = annotator.annotate('audio.wav')
   ```

3. **Cleanup**:
   ```bash
   sky down sky-33ba-marcos
   ```

4. **Then run**:
   - Multi-GPU test (optional)
   - OR Full Orpheus annotation (118k samples)

---

## 📈 Progress Updates

### ✅ Job Completed - Partial Success

**Time**: 2025-12-02 13:03 UTC
**Duration**: 3 minutes
**Status**: ✅ SUCCEEDED (com erro no model loading)

#### What Worked ✅
- ✅ Machine provisioned (A100 SXM4, 32 vCPUs, 64GB RAM)
- ✅ Dependencies installed (torch, transformers, librosa)
- ✅ Repository cloned
- ✅ **350 synthetic samples created** (50/emotion)
- ✅ **Dataset prepared** (data/prepared/synthetic_prepared)

#### Issues Found ❌
- ❌ emotion2vec model loading failed
- ❌ Model requires `funasr` library (not standard transformers)
- ❌ Fine-tuning didn't execute
- ❌ Model testing failed

#### Next Steps 🔧
1. Update emotion2vec implementation to use compatible wav2vec2
2. Re-run fine-tuning with corrected code
3. Or: Install funasr for native emotion2vec support

**Last update**: 2025-12-02 13:07 UTC - Completed with model loading error