End of training

README.md

@@ -104,7 +104,7 @@ xformers_attention: null
 
 This model is a fine-tuned version of [EleutherAI/pythia-14m](https://huggingface.co/EleutherAI/pythia-14m) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 7.4194
+- Loss: 7.5607
 
 ## Model description
 
@@ -139,9 +139,9 @@ The following hyperparameters were used during training:
 | Training Loss | Epoch  | Step | Validation Loss |
 |:-------------:|:------:|:----:|:---------------:|
 | 42.0391       | 0.0013 | 1    | 10.8013         |
-| 40.9297       | 0.0040 | 3    | 10.4305         |
-| 39.2031       | 0.0079 | 6    | 8.9850          |
-| 30.0859       | 0.0119 | 9    | 7.4194          |
+| 42.5078       | 0.0040 | 3    | 10.4134         |
+| 44.0547       | 0.0079 | 6    | 9.3822          |
+| 30.1562       | 0.0119 | 9    | 7.5607          |
 
 
 ### Framework versions

adapter_config.json

@@ -20,10 +20,10 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
+    "dense_h_to_4h",
     "dense",
-    "query_key_value",
     "dense_4h_to_h",
-    "dense_h_to_4h"
+    "query_key_value"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

adapter_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:36283aa446e5062a903eb43b2ce1ab305aa3809cfbbcbd93dc0d8690076b0448
+oid sha256:b60b7aad1f1f52839a373900df7cc27805d8e84caf057397c23b5491d74dbd39
 size 804030

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2cdd41b8034aebed5f4cfddad21d3fd7abe661ba76b481044ea408ec6239fa06
+oid sha256:55047a50e41be0e4e6dd996b8c2ea294dd61c14b8f6b1fe8713482713f5c79f2
 size 792912

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:097bd66dc3d47f5850103823e5f488e05d11aad4916f7f9c6e0b797fc472f3b8
+oid sha256:4a8fc328665b58033bd97cc1ce9c1a9a184f8df499d10df35acdb35b8dc41988
 size 6776