Model Stock: All we need is just a few fine-tuned models
Paper
•
2403.19522
•
Published
•
13
modelstock
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Method
This model was merged using the Model Stock merge method using /alloc/axolotl/Rae-15B-chat/checkpoint-1568 as a base.
Models Merged
The following models were included in the merge:
- /alloc/axolotl/Rae-15B-chat/checkpoint-1470
- /alloc/axolotl/Rae-15B-chat/checkpoint-1274
- /alloc/axolotl/Rae-15B-chat/checkpoint-1372
Configuration
The following YAML configuration was used to produce this model:
models:
- model: /alloc/axolotl/Rae-15B-chat/checkpoint-1372
- model: /alloc/axolotl/Rae-15B-chat/checkpoint-1470
- model: /alloc/axolotl/Rae-15B-chat/checkpoint-1274
merge_method: model_stock
base_model: /alloc/axolotl/Rae-15B-chat/checkpoint-1568
normalize: true
int8_mask: true
dtype: bfloat16
- Downloads last month
- 1