Sparrow

麻雀虽小 五脏俱全

Small as it is, the sparrow has all the vital organs

For more detail pleace check: Sparrow

To run the model, first download the code from huggingface:

git clone https://huggingface.co/TerenceLau/sparrow && cd ./sparrow/

then, you can register the SparrowConfig and SparrowModel to AutoConfig and AutoModelForCausalLM:

from modelling_sparrow import SparrowModelForCausalLM, SparrowConfig
from transformers import AutoConfig, AutoModelForCausalLM, AutoTokenizer
AutoConfig.register("sparrow", SparrowConfig)
AutoModelForCausalLM.register(SparrowConfig, SparrowModelForCausalLM)
model = AutoModelForCausalLM.from_pretrained("TerenceLau/sparrow")