1 contributor

History: 12 commits

SQCU

Upload ascii-chart5-L4-D768-mkii-c1932d6c-1962-493d-b0b7-78e84e30e4e5.txt with huggingface_hub

4adce2e verified 4 months ago

ascii-chart5-L4-D768-mkii-c1932d6c-1962-493d-b0b7-78e84e30e4e5
Upload folder using huggingface_hub 4 months ago
ascii-chart5-L4-D768-mkiii-765ee0b7-5a50-45a0-abcf-2c95a75feab7
Upload folder using huggingface_hub 4 months ago
ascii-eos-L4-D768-rollout-test-01661b5f-2ff7-49b1-9ad8-fee77e14bd1c
Upload folder using huggingface_hub 4 months ago
basic_pgptlformer-e118dbaf-93da-4b79-8ab7-ed696e2d80bc
Upload 8 files about 1 year ago
dyn_qkl2norm-f68a7719-e9c8-4329-8177-19a229adf90c
Upload 8 files about 1 year ago
dyn_qkrmsnorm-3824bd2b-dd4b-4737-b074-57574f2cd8fc
Upload 8 files about 1 year ago
dyn_qkrmsnorm_ii-7a038ecd-be98-46cb-abe8-e0f013fd7eed
sling the illustrious and mysterious "attention_II" models. also some layerwise rmsnorm, qkprojection rmsnorm models, one twice as large as the other. about 1 year ago
qkrmsnorm-a283dcc5-027f-4618-8e8f-460bef6b2739
Upload 8 files about 1 year ago
re-pqt-rmsXrms-2fd4a7cb-930a-464b-90e0-eef9d7551d8d
sling the illustrious and mysterious "attention_II" models. also some layerwise rmsnorm, qkprojection rmsnorm models, one twice as large as the other. about 1 year ago
re-pqt-rmsXrms-2x-42e14b65-2277-45ae-a68c-822eb66be09a
sling the illustrious and mysterious "attention_II" models. also some layerwise rmsnorm, qkprojection rmsnorm models, one twice as large as the other. about 1 year ago
re-pqt-rmsXrms-ATTNII-697f0113-bb05-480b-b6dc-42a97de0de3e
sling the illustrious and mysterious "attention_II" models. also some layerwise rmsnorm, qkprojection rmsnorm models, one twice as large as the other. about 1 year ago
re-pqt-rmsXrms-ATTNII_rev2-d46b31ce-94f1-41c7-89cb-358a7a3f316b
compiled models train faster so you can train more of them in a short experiment, to better convergence. about 1 year ago
re-pqt-rmsXrmsx2x2-ATTNII-791967c5-5c59-4a5f-a2c5-07772bcf65ab
89,301,000 parameter attention_ii, z_lossed model trained for 6250 steps at batchsize:4*32, device_batchsize:32 about 1 year ago
re-pqt-rmsXrmsx3-ATTNII_fast-af1c5037-28b9-4ce3-b351-c78046f90ee7
compiled models train faster so you can train more of them in a short experiment, to better convergence. about 1 year ago
.gitattributes

1.52 kB

initial commit about 1 year ago
README.md

712 Bytes

Update README.md about 1 year ago
ascii-chart5-L4-D768-mkii-c1932d6c-1962-493d-b0b7-78e84e30e4e5.txt

214 kB

Upload ascii-chart5-L4-D768-mkii-c1932d6c-1962-493d-b0b7-78e84e30e4e5.txt with huggingface_hub 4 months ago
ascii-eos-L4-D768-rollout-test-01661b5f-2ff7-49b1-9ad8-fee77e14bd1c.txt

250 kB

Upload ascii-eos-L4-D768-rollout-test-01661b5f-2ff7-49b1-9ad8-fee77e14bd1c.txt with huggingface_hub 4 months ago
dyn_qkl2norm-f68a7719-e9c8-4329-8177-19a229adf90c.txt

474 kB

Upload 8 files about 1 year ago
dyn_qkrmsnorm-3824bd2b-dd4b-4737-b074-57574f2cd8fc.txt

474 kB

Upload 8 files about 1 year ago
dyn_qkrmsnorm_ii-7a038ecd-be98-46cb-abe8-e0f013fd7eed.txt

474 kB

sling the illustrious and mysterious "attention_II" models. also some layerwise rmsnorm, qkprojection rmsnorm models, one twice as large as the other. about 1 year ago
e118dbaf-93da-4b79-8ab7-ed696e2d80bc.txt

474 kB

Upload 8 files about 1 year ago
qkrmsnorm-a283dcc5-027f-4618-8e8f-460bef6b2739.txt

473 kB

Upload 8 files about 1 year ago
re-pqt-rmsXrms-2fd4a7cb-930a-464b-90e0-eef9d7551d8d.txt

474 kB

sling the illustrious and mysterious "attention_II" models. also some layerwise rmsnorm, qkprojection rmsnorm models, one twice as large as the other. about 1 year ago
re-pqt-rmsXrms-2x-42e14b65-2277-45ae-a68c-822eb66be09a.txt

474 kB

sling the illustrious and mysterious "attention_II" models. also some layerwise rmsnorm, qkprojection rmsnorm models, one twice as large as the other. about 1 year ago
re-pqt-rmsXrms-ATTNII-697f0113-bb05-480b-b6dc-42a97de0de3e.txt

576 kB

sling the illustrious and mysterious "attention_II" models. also some layerwise rmsnorm, qkprojection rmsnorm models, one twice as large as the other. about 1 year ago
re-pqt-rmsXrms-ATTNII_rev2-d46b31ce-94f1-41c7-89cb-358a7a3f316b.txt

570 kB

compiled models train faster so you can train more of them in a short experiment, to better convergence. about 1 year ago
re-pqt-rmsXrmsx2x2-ATTNII-791967c5-5c59-4a5f-a2c5-07772bcf65ab.txt

585 kB

89,301,000 parameter attention_ii, z_lossed model trained for 6250 steps at batchsize:4*32, device_batchsize:32 about 1 year ago
re-pqt-rmsXrmsx3-ATTNII_fast-af1c5037-28b9-4ce3-b351-c78046f90ee7.txt

3.72 MB

compiled models train faster so you can train more of them in a short experiment, to better convergence. about 1 year ago