VORTEX's picture

Building on HF

VORTEX

Abhaykoul

·

OEvortex

AI & ML interests

None yet

Recent Activity

updated a model 11 days ago

Abhaykoul/ARC-HAIRM

published a model 11 days ago

Abhaykoul/ARC-HAIRM

reacted to KingNish's post with 🔥 16 days ago

Muon vs MuonClip vs Muon+Adamw Muon has gone from an experiment to a mainstream optimizer, but does it hold up for fine‑tuning? We ran head‑to‑head tests on Qwen3‑4B (10k+ high‑quality instruction rows) to find out. Short story: Pure Muon converged fastest at the start, but its gradient‑norm spikes made training unstable. MuonClip (Kimi K2’s clipping) stabilizes long pretraining runs, yet in our small‑scale fine‑tune it underperformed, lower token accuracy and slower convergence. The winner was the hybrid: Muon for 2D layers + AdamW for 1D layers. It delivered the best balance of stability and final performance and even beat vanilla AdamW. Takeaway: for small-scale fine-tuning, hybrid = practical and reliable. Next Step: scale to larger models/datasets to see if Muon’s spikes become catastrophic or if clipping wins out. Full Blog Link: https://huggingface.co/blog/KingNish/optimizer-part1

View all activity

Organizations

New activity in Omarrran/Kashmiri_Text_Corpus_Cleaned_2025_HNM 27 days ago

Can i get its access?

#2 opened 3 months ago by

New activity in UnfilteredAI/DAN 2 months ago

Update README.md

#2 opened 2 months ago by

New activity in IndianAIDevs/README 2 months ago

General

#5 opened 3 months ago by

New activity in IndianAIDevs/README 3 months ago

Let's Talk about AI

#1 opened 4 months ago by

New activity in HelpingAI/HELVETE-3B 4 months ago

architecture

#7 opened 9 months ago by

New activity in UnfilteredAI/NSFW-gen-v2 4 months ago

Create nsfw roleplay

#21 opened 4 months ago by

New activity in HelpingAI/hai3.1-checkpoint-0002 4 months ago

remove dupe

#2 opened 5 months ago by

New activity in HelpingAI/HELVETE-3B 5 months ago

this is not doing much except whatever it wants

#1 opened over 1 year ago by

New activity in HelpingAI/hai3.1-checkpoint-0002 5 months ago

Update README.md

#1 opened 5 months ago by

New activity in HelpingAI/Dhanishtha-nsfw 5 months ago

Update README.md

#2 opened 5 months ago by

Update README.md

#1 opened 5 months ago by

New activity in HelpingAI/HelpingAI2.5-10B 6 months ago

Base model?

#1 opened 6 months ago by

New activity in HelpingAI/Dhanishtha-2.0-preview 6 months ago

long Context length

#1 opened 6 months ago by

New activity in HelpingAI/Dhanishtha-2.0-preview 6 months ago

Apply for community grant: Company project (gpu)

#1 opened 6 months ago by

New activity in garage-lab/MCP_HTML2JSON 6 months ago

bug in mcp

#1 opened 6 months ago by

New activity in DontPlanToEnd/UGI-Leaderboard 6 months ago

[Model Evaluation] HELVETE-X

#302 opened 6 months ago by

New activity in UnfilteredAI/DAN-L3-R1-8B 7 months ago

Request: DOI

#2 opened 7 months ago by

New activity in huggingface/InferenceSupport 8 months ago

UnfilteredAI/DAN-L3-R1-8B

#1772 opened 8 months ago by

New activity in UnfilteredAI/DAN-Qwen3-1.7B 8 months ago

Adding `safetensors` variant of this model

#1 opened 8 months ago by

New activity in OEvortex/TTS-OLD 8 months ago

Seeking help: TypeError: DacModel.decode() missing 1 required positional argument: 'quantized_representation'

#3 opened 8 months ago by