sherryxychen/2025-10-13_smolvla4
Robotics
•
Updated
•
2
Sherry Chen PRO
sherryxychen
AI & ML interests
Learning-based robotics, VLA
Recent Activity
updated
a model
about 1 month ago
sherryxychen/2025-10-13_smolvla4
published
a model
about 1 month ago
sherryxychen/2025-10-13_smolvla4
updated
a model
about 1 month ago
sherryxychen/2025-10-13_test-smolvla3
Organizations
None yet
updated
a
model
about 1 month ago
published
a
model
about 1 month ago
updated
a
model
about 1 month ago
published
a
model
about 1 month ago
updated
a
model
about 1 month ago
published
a
model
about 1 month ago
updated
a
model
about 1 month ago
published
a
model
about 1 month ago
posted
an
update
about 2 months ago
Post
368
**Training ACT on SO-101: From Woodpecker to 90% Success (All the Mistakes Included)**
I spent 3 weeks training Action Chunking Transformer on SO-101 for pick-and-place. Spoiler: the first attempt trained a woodpecker that just pecked the table. 🐦
**What's Different About This Post:**
Most ACT tutorials show the success. I documented every failure, hardware issue, and debugging step. If you're new to SO-101/LeRobot/ACT, hopefully my mistakes save you time.
Try 1: The Woodpecker
- Followed the LeRobot tutorial, collected 50 episodes
- Beautiful loss curves ✅
- Robot learned to peck at table ❌
- Rookie mistakes: moving cameras, arm calibration mismatch, limited data diversity, looking at follower arm during teleop (it's cheating!)
Try 2: Engineering Upgrades
- Fixed hardware setup (tape + markers everywhere)
- USB udev rules for camera stability
- Formal task definition with stratified sampling
- Built proper eval pipeline with progress scoring
- Motor breakdown mid-collection (broke the gripper with excessive force 💀)
- Results: 60% in-distribution success, 10% OOD (better, but not great)
Try 3: More & Better Data
sherryxychen/2025-09-07_act
- 125 episodes with rotation variations ( sherryxychen/2025-09-01_pick-and-place-block)
- Better workspace coverage
- Improved grasping technique
- Results: 90% in-distribution success, 75% OOD! 🎉
Key Learnings:
- Consistent hardware setup is everything
- Don't look at the follower arm during teleop
- Data diversity is key for generalization
- Debug infrastructure matters
- Real robots break in mysterious ways (buy spare motors!)
Full write-up (with more videos!): https://huggingface.co/blog/sherryxychen/train-act-on-so-101
Code: https://github.com/sherrychen1120/so101_bench
Happy to answer any questions! 🤗
#imitation-learning #lerobot #act #so-101
I spent 3 weeks training Action Chunking Transformer on SO-101 for pick-and-place. Spoiler: the first attempt trained a woodpecker that just pecked the table. 🐦
**What's Different About This Post:**
Most ACT tutorials show the success. I documented every failure, hardware issue, and debugging step. If you're new to SO-101/LeRobot/ACT, hopefully my mistakes save you time.
Try 1: The Woodpecker
- Followed the LeRobot tutorial, collected 50 episodes
- Beautiful loss curves ✅
- Robot learned to peck at table ❌
- Rookie mistakes: moving cameras, arm calibration mismatch, limited data diversity, looking at follower arm during teleop (it's cheating!)
Try 2: Engineering Upgrades
- Fixed hardware setup (tape + markers everywhere)
- USB udev rules for camera stability
- Formal task definition with stratified sampling
- Built proper eval pipeline with progress scoring
- Motor breakdown mid-collection (broke the gripper with excessive force 💀)
- Results: 60% in-distribution success, 10% OOD (better, but not great)
Try 3: More & Better Data
sherryxychen/2025-09-07_act
- 125 episodes with rotation variations ( sherryxychen/2025-09-01_pick-and-place-block)
- Better workspace coverage
- Improved grasping technique
- Results: 90% in-distribution success, 75% OOD! 🎉
Key Learnings:
- Consistent hardware setup is everything
- Don't look at the follower arm during teleop
- Data diversity is key for generalization
- Debug infrastructure matters
- Real robots break in mysterious ways (buy spare motors!)
Full write-up (with more videos!): https://huggingface.co/blog/sherryxychen/train-act-on-so-101
Code: https://github.com/sherrychen1120/so101_bench
Happy to answer any questions! 🤗
#imitation-learning #lerobot #act #so-101
published
an
article
about 2 months ago
Article
How I Trained Action Chunking Transformer (ACT) on SO-101: My Journey, Gotchas, and Lessons
•
40
upvoted
an
article
2 months ago
Article
SmolVLA: Efficient Vision-Language-Action Model trained on Lerobot Community Data
•
284
updated
a
model
3 months ago
published
a
model
3 months ago
updated
a
dataset
3 months ago
updated
a
model
3 months ago
published
a
model
3 months ago
published
a
dataset
3 months ago
updated
a
dataset
4 months ago
published
a
dataset
4 months ago
updated
a
model
4 months ago