xXFiEsTaDeAmOnXx commited on
Commit
04a7639
·
verified ·
1 Parent(s): b3c7342

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -2
README.md CHANGED
@@ -12,9 +12,16 @@ tags:
12
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
13
 
14
  ## Merge Details
15
- ### Merge Method
16
 
17
- This model was merged using the Passthrough merge method.
 
 
 
 
 
 
 
 
18
 
19
  ### Models Merged
20
 
 
12
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
13
 
14
  ## Merge Details
 
15
 
16
+ The model was merged using the **Passthrough merge method**, with a **sliding window approach**.
17
+
18
+ ### Duplicated Layers
19
+
20
+ Due to the sliding window method, layers overlap between slices. The following layers are duplicated across multiple slices:
21
+
22
+ - Layers 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32
23
+
24
+ This means each layer is used in multiple slices, with each slice overlapping with the previous and next slice.
25
 
26
  ### Models Merged
27