PT/SFT/RLHF/GRPO on ERNIE Supported with MS-SWIFT
#7
by
tastelikefeet
- opened
You can use any examples in: https://github.com/modelscope/ms-swift/tree/main/examples/train
with model-id:
--model baidu/ERNIE-4.5-21B-A3B-Thinking \
--use_hf true
or with modelscope:
--model PaddlePaddle/ERNIE-4.5-21B-A3B-Thinking \
--use_hf false
Meanwhile, we recommand training with Megatron, this will be 10 times faster for MoE models:
megatron sft \
...
Check these scripts:
https://github.com/modelscope/ms-swift/tree/main/examples/megatron