Text Generation
Safetensors
gpt_neox
DiamondGotCat commited on
Commit
e02341c
·
verified ·
1 Parent(s): 7af8936

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +36 -1
README.md CHANGED
@@ -7,4 +7,39 @@ base_model:
7
  pipeline_tag: text-generation
8
  ---
9
 
10
- GitHub: [Zeta](https://github.com/DiamondGotCat/Zeta)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7
  pipeline_tag: text-generation
8
  ---
9
 
10
+ # Zeta 1 - First Step of Zeta Project, This is Epoch.
11
+ Zeta 1 is the first step that I, an ordinary person and consumer, have taken in pursuit of my dream as an individual.
12
+
13
+ ## About Zeta 1
14
+ Zeta 1 is an LLM with about 400 million parameters.
15
+
16
+ It might be better to call it an SLM.
17
+
18
+ Zeta 1 is an SLM that was painstakingly created on a consumer computer.
19
+
20
+ ### Computer Spec
21
+
22
+ **Machine:** *Mac Mini (M2 Pro, 32GB RAM, 2023)*
23
+
24
+ This is a technical matter, but on a Mac with Apple Silicon you can only choose between CPU or MPS (Apple's proprietary API).
25
+
26
+ CPU has some compatibility but is too slow.
27
+
28
+ MPS is somewhat faster but doesn't allow the use of optimization systems such as fp16.
29
+
30
+ Zeta 1 was built using MPS.
31
+
32
+ ### Arguments of Trainer
33
+ - **train epochs:** 3
34
+ - **warmup steps:** 100
35
+
36
+ ### Datasets
37
+ Details of the dataset used can be found [here](https://github.com/DiamondGotCat/Zeta-Dataset/releases/tag/zeta-1)
38
+
39
+ ## Links
40
+ GitHub: [Zeta](https://github.com/DiamondGotCat/Zeta)
41
+
42
+ ---
43
+
44
+ Zeta is just a small SLM.
45
+ But don't forget that it has big dreams inside.