River-gut-celeste-12b-della / README.md

Ennthen

Upload folder using huggingface_hub

56d9f70 verified 3 months ago

preview code

raw

history blame contribute delete

1.13 kB

metadata

license: apache-2.0
tags:
  - merge
  - mergekit
  - lazymergekit
  - TheDrummer/Rivermind-Lux-12B-v1
  - nbeerbower/mistral-nemo-gutenberg-12B-v4
  - nothingiisreal/MN-12B-Celeste-V1.9

River-gut-celeste-12b-della

River-gut-celeste-12b-della is a merge of the following models using mergekit:

🧩 Configuration

merge_method: della
dtype: bfloat16
out_dtype: bfloat16
base_model: HumanLLMs/Human-Like-Mistral-Nemo-Instruct-2407
models:
  - model: TheDrummer/Rivermind-Lux-12B-v1
    parameters:
      density: 0.6
      weight: 0.7
  - model: nbeerbower/mistral-nemo-gutenberg-12B-v4
    parameters:
      density: 0.6
      weight: 0.5
  - model: nothingiisreal/MN-12B-Celeste-V1.9
    parameters:
      density: 0.5
      weight: 0.3
parameters:
  epsilon: 0.1
  lambda: 1.0