lbourdois commited on
Commit
8c41b7d
·
verified ·
1 Parent(s): 5e5bdd7

Improve language tag

Browse files

Hi! As the model is multilingual, this is a PR to add other languages than English to the language tag to improve the referencing. Note that 29 languages are announced in the README, but only 13 are explicitly listed. I was therefore only able to add these 13 languages.

Files changed (1) hide show
  1. README.md +68 -55
README.md CHANGED
@@ -1,55 +1,68 @@
1
- ---
2
- base_model:
3
- - scb10x/typhoon2-qwen2.5-7b-instruct
4
- - deepseek-ai/DeepSeek-R1-Distill-Qwen-7B
5
- - Qwen/Qwen2.5-7B-Instruct
6
- - Qwen/Qwen2.5-7B
7
- library_name: transformers
8
- tags:
9
- - mergekit
10
- - merge
11
-
12
- ---
13
- # merge
14
-
15
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
16
-
17
- ## Merge Details
18
- ### Merge Method
19
-
20
- This model was merged using the [Linear DARE](https://arxiv.org/abs/2311.03099) merge method using [Qwen/Qwen2.5-7B](https://huggingface.co/Qwen/Qwen2.5-7B) as a base.
21
-
22
- ### Models Merged
23
-
24
- The following models were included in the merge:
25
- * [scb10x/typhoon2-qwen2.5-7b-instruct](https://huggingface.co/scb10x/typhoon2-qwen2.5-7b-instruct)
26
- * [deepseek-ai/DeepSeek-R1-Distill-Qwen-7B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B)
27
- * [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct)
28
-
29
- ### Configuration
30
-
31
- The following YAML configuration was used to produce this model:
32
-
33
- ```yaml
34
- models:
35
- - model: Qwen/Qwen2.5-7B
36
- - model: scb10x/typhoon2-qwen2.5-7b-instruct
37
- parameters:
38
- density: 0.80
39
- weight: [0.2, 0.2, 0.5, 0.8]
40
- - model: deepseek-ai/DeepSeek-R1-Distill-Qwen-7B
41
- parameters:
42
- density: 0.50
43
- weight: [0.1, 0.5, 0.5, 0.2]
44
- - model: Qwen/Qwen2.5-7B-Instruct
45
- parameters:
46
- density: 0.2
47
- weight: [0.7, 0.3, 0.0, 0.0]
48
- merge_method: dare_linear
49
- base_model: Qwen/Qwen2.5-7B
50
- parameters:
51
- normalize: true
52
- dtype: bfloat16
53
- tokenizer:
54
- source: scb10x/typhoon2-qwen2.5-7b-instruct
55
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - scb10x/typhoon2-qwen2.5-7b-instruct
4
+ - deepseek-ai/DeepSeek-R1-Distill-Qwen-7B
5
+ - Qwen/Qwen2.5-7B-Instruct
6
+ - Qwen/Qwen2.5-7B
7
+ library_name: transformers
8
+ tags:
9
+ - mergekit
10
+ - merge
11
+ language:
12
+ - zho
13
+ - eng
14
+ - fra
15
+ - spa
16
+ - por
17
+ - deu
18
+ - ita
19
+ - rus
20
+ - jpn
21
+ - kor
22
+ - vie
23
+ - tha
24
+ - ara
25
+ ---
26
+ # merge
27
+
28
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
29
+
30
+ ## Merge Details
31
+ ### Merge Method
32
+
33
+ This model was merged using the [Linear DARE](https://arxiv.org/abs/2311.03099) merge method using [Qwen/Qwen2.5-7B](https://huggingface.co/Qwen/Qwen2.5-7B) as a base.
34
+
35
+ ### Models Merged
36
+
37
+ The following models were included in the merge:
38
+ * [scb10x/typhoon2-qwen2.5-7b-instruct](https://huggingface.co/scb10x/typhoon2-qwen2.5-7b-instruct)
39
+ * [deepseek-ai/DeepSeek-R1-Distill-Qwen-7B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B)
40
+ * [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct)
41
+
42
+ ### Configuration
43
+
44
+ The following YAML configuration was used to produce this model:
45
+
46
+ ```yaml
47
+ models:
48
+ - model: Qwen/Qwen2.5-7B
49
+ - model: scb10x/typhoon2-qwen2.5-7b-instruct
50
+ parameters:
51
+ density: 0.80
52
+ weight: [0.2, 0.2, 0.5, 0.8]
53
+ - model: deepseek-ai/DeepSeek-R1-Distill-Qwen-7B
54
+ parameters:
55
+ density: 0.50
56
+ weight: [0.1, 0.5, 0.5, 0.2]
57
+ - model: Qwen/Qwen2.5-7B-Instruct
58
+ parameters:
59
+ density: 0.2
60
+ weight: [0.7, 0.3, 0.0, 0.0]
61
+ merge_method: dare_linear
62
+ base_model: Qwen/Qwen2.5-7B
63
+ parameters:
64
+ normalize: true
65
+ dtype: bfloat16
66
+ tokenizer:
67
+ source: scb10x/typhoon2-qwen2.5-7b-instruct
68
+ ```