Text Generation
Safetensors
qwen2
conversational
DiamondGotCat commited on
Commit
6fb70e0
·
verified ·
1 Parent(s): 83df005

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +59 -0
README.md CHANGED
@@ -1,3 +1,62 @@
1
  ---
2
  license: mit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: mit
3
+ language:
4
+ - zho
5
+ - eng
6
+ - fra
7
+ - spa
8
+ - por
9
+ - deu
10
+ - ita
11
+ - rus
12
+ - jpn
13
+ - kor
14
+ - vie
15
+ - tha
16
+ - ara
17
+ base_model:
18
+ - Qwen/Qwen2.5-0.5B-Instruct
19
+ pipeline_tag: text-generation
20
+ datasets:
21
+ - DiamondGotCat/Zeta-2-Dataset
22
  ---
23
+
24
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/667e7043f6fa6fb03b220f9c/SZ4cb7QP95DmPTYIFe8r7.png)
25
+
26
+ # Further into the future. Better. New achievement: Zeta 2.
27
+ Zeta 2 is a new version of Zeta that aims to take the next step forward from Zeta 1, which proved that "ordinary consumers can create LLMs."
28
+
29
+ ## About Zeta 2
30
+ Zeta 2 is an LLM with about 460 million parameters.
31
+
32
+ It might be better to call it an SLM.
33
+
34
+ Zeta 2 is an SLM that was painstakingly created on a consumer computer.
35
+
36
+ ### Computer Spec
37
+
38
+ **Machine:** *Mac Mini (M2 Pro, 32GB RAM, 2023)*
39
+ **Issues:** *Training failed twice due to lack of RAM*
40
+
41
+ This is a technical matter, but on a Mac with Apple Silicon you can only choose between CPU or MPS (Apple's proprietary API).
42
+
43
+ CPU has some compatibility but is too slow.
44
+
45
+ MPS is somewhat faster but doesn't allow the use of optimization systems such as fp16.
46
+
47
+ Zeta 2 was built using MPS.
48
+
49
+ ### Arguments of Trainer
50
+ - **train epochs:** 3
51
+ - **warmup steps:** 100
52
+
53
+ ### Datasets
54
+ Details of the dataset used can be found [here](https://github.com/DiamondGotCat/Zeta-Dataset/releases/tag/zeta-2)
55
+
56
+ ## Links
57
+ GitHub: [Zeta](https://github.com/DiamondGotCat/Zeta)
58
+
59
+ ---
60
+
61
+ Zeta is just a small SLM.
62
+ But don't forget that it has big dreams inside.