Update README.md
Browse files
README.md
CHANGED
|
@@ -111,4 +111,16 @@ Predicted Tokens
|
|
| 111 |
* **Optimizer:** AdamW or Lion (4-bit / mixed precision)
|
| 112 |
* **Dataset:** TinyChat (~50–200 MB)
|
| 113 |
* **Training loop:** gradient checkpointing + recomputation
|
| 114 |
-
* **Objective:** token prediction / text generation
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 111 |
* **Optimizer:** AdamW or Lion (4-bit / mixed precision)
|
| 112 |
* **Dataset:** TinyChat (~50–200 MB)
|
| 113 |
* **Training loop:** gradient checkpointing + recomputation
|
| 114 |
+
* **Objective:** token prediction / text generation
|
| 115 |
+
|
| 116 |
+
|
| 117 |
+
## Citation
|
| 118 |
+
|
| 119 |
+
```bibtex
|
| 120 |
+
@software{lorpt2025,
|
| 121 |
+
title={LoRPt: Low-Rank Pretraining for Resource-Efficient Language Models},
|
| 122 |
+
author={[FlameF0X]},
|
| 123 |
+
year={2025},
|
| 124 |
+
url={https://github.com/FlameF0X/Low-Rank-Pretraining}
|
| 125 |
+
}
|
| 126 |
+
```
|