alisawuffles/superbpe-tokenizer-128k
Updated
SuperBPE
SuperBPE tokenizers and models trained with them
Note Recommended starting point for experimentation in your own model development pipeline!
-
6
SuperBPE Playground
📝Experiment with and compare different tokenizers
-
SuperBPE: Space Travel for Language Models
Paper • 2503.13423 • Published • 13 -
UW/OLMo2-8B-SuperBPE-t180k
Text Generation • 8B • Updated • 42 • 8 -
UW/OLMo2-11B-SuperBPE-t180k
Text Generation • 11B • Updated • 5 • 2 -
UW/OLMo2-8B-SuperBPE-t160k
Text Generation • 8B • Updated • 4 • 3 -
UW/OLMo2-8B-SuperBPE-t80k
Text Generation • 8B • Updated • 1 -
UW/OLMo2-8B-BPE
Text Generation • 8B • Updated • 2 -
UW/olmo-mix-1124-subset-p99
Updated • 850 • 1 -
alisawuffles/bpe-tokenizer-128k
Updated