shuttleai/shuttle-3.5-moe-ckpts
Updated
•
5
ShuttleAI
company
Verified
AI & ML interests
None defined yet.
Update README.md
#9 opened 7 months ago
by
minpeter
Update README.md
#10 opened 7 months ago
by
minpeter
Update README.md
#7 opened 7 months ago
by
minpeter
Update README.md
#8 opened 7 months ago
by
minpeter
xtristan
updated
a
model
8 months ago
🚩 Report: Legal issue(s)
🚀
2
3
#3 opened 8 months ago
by
llama-anon
xtristan
published
a
model
8 months ago
ollama ready version please
3
#1 opened 8 months ago
by
ryg81
xtristan
updated
a
model
8 months ago
xtristan
updated
a
collection
8 months ago
xtristan
published
a
model
8 months ago
xtristan
updated
a
model
8 months ago
xtristan
published
a
model
8 months ago
e-n-v-y
updated
a
model
9 months ago
Alignment-Lab-AI
posted
an
update
about 1 year ago
Post
1855
remember boys and girls, always keep all your data, its never a waste of time!
Fizzarolli
posted
an
update
over 1 year ago
Post
2846
hi everyone!
i wanted to share an experiment i did with upcycling phi-3 mini into an moe recently.
while benchmarks are definitely within a margin of error and they performed similarly, i think it's an interesting base to try and see if you can improve phi's performance! (maybe looking into HuggingFaceFW/fineweb-edu could be interesting, i also left some other notes if anyone with more compute access wants to try it themselves)
check it out! Fizzarolli/phi3-4x4b-v1
i wanted to share an experiment i did with upcycling phi-3 mini into an moe recently.
while benchmarks are definitely within a margin of error and they performed similarly, i think it's an interesting base to try and see if you can improve phi's performance! (maybe looking into HuggingFaceFW/fineweb-edu could be interesting, i also left some other notes if anyone with more compute access wants to try it themselves)
check it out! Fizzarolli/phi3-4x4b-v1
Fizzarolli
posted
an
update
over 1 year ago
Post
3261
Is anyone looking into some sort of decentralized/federated dataset generation or classification by humans instead of synthetically?
From my experience with trying models, a *lot* of modern finetunes are trained on what amounts to, in essence, GPT-4 generated slop that makes everything sound like a rip-off GPT-4 (refer to i.e. the Dolphin finetunes). I have a feeling that this is a lot of the reason people haven't been quite as successful as Meta's instruct tunes of Llama 3.
From my experience with trying models, a *lot* of modern finetunes are trained on what amounts to, in essence, GPT-4 generated slop that makes everything sound like a rip-off GPT-4 (refer to i.e. the Dolphin finetunes). I have a feeling that this is a lot of the reason people haven't been quite as successful as Meta's instruct tunes of Llama 3.