Softpick
Collection
Pretrained models from the paper "Softpick: No Attention Sink, No Massive Activations with Rectified Softmax"
•
5 items
•
Updated
YAML Metadata
Warning:
empty or missing yaml metadata in repo card
(https://huggingface.co/docs/hub/model-cards#model-card-metadata)
This model is from the paper arxiv.org/abs/2504.20966
Softpick: No Attention Sink, No Massive Activations with Rectified Softmax
Also used in arxiv.org/abs/2508.19228
Token Order Prediction
See code: https://github.com/zaydzuhri/softpick-attention
This model is only usable through these repositories: https://github.com/zaydzuhri/flash-linear-attention/tree/softpick-attention https://github.com/zaydzuhri/flame/tree/softpick-attention
- Downloads last month
- 16
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
🙋
Ask for provider support