MegaMoon-Karcher-12B

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the Karcher Mean merge method.

Models Merged

The following models were included in the merge:

Configuration

The following YAML configuration was used to produce this model:


models:
  - model: LatitudeGames/Muse-12B
  - model: anthracite-org/magnum-v4-12b
  - model: yamatazen/NeonMaid-12B-v2
  - model: SicariusSicariiStuff/Impish_Nemo_12B
  - model: crestf411/MN-Slush
  - model: Epiculous/Violet_Twilight-v0.2
  - model: LatitudeGames/Wayfarer-12B
  - model: inflatebot/MN-12B-Mag-Mell-R1
  - model: nothingiisreal/MN-12B-Celeste-V1.9
  - model: Nitral-AI/Captain-Eris_Violet-V0.420-12B
  - model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b  
  - model: Vortex5/Poetic-Rune-12B
  - model: Vortex5/Lunar-Nexus-12B
  - model: Vortex5/Harmonic-Moon-12B
  - model: Vortex5/Crystal-Moon-12B
  - model: Vortex5/Lunar-Nexus-12B
  - model: Vortex5/Moonlit-Shadow-12B
merge_method: karcher
dtype: bfloat16
parameters:
  tol: 1e-10
  max_iter: 999
tokenizer:
  source: union