Growing Transformers: Modular Composition and Layer-wise Expansion on a Frozen Substrate
Paper
•
2507.07129
•
Published
•
3
Andrey
Bochkov
AI & ML interests
None yet
Recent Activity
reacted
to
sergiopaniego's
post
with 🔥
16 days ago
New REPL environment in OpenEnv available! ✨
Used in the Recursive Language Models (RLM) paper by Alex Zhang.
Ready for inference & post-training using trajectories. Handles long contexts:
> Run Python code in a sandbox
> Make recursive calls to LMs
> Explore data programmatically
> Return final result
Docs: https://meta-pytorch.org/OpenEnv/environments/repl/
Inference script: https://github.com/meta-pytorch/OpenEnv/blob/main/examples/repl_oolong_simple.py
updated
a model
20 days ago
Bochkov/growing-transformers-model-frozen-16-bit-baseline-monolyth-181m
Organizations
None yet