training.log

B:\8B\!models--SicariusSicariiStuff--Llama-3.1-Nemotron-8B-UltraLong-1M-Instruct_Abliterated>PMPF4_finetune_7B_3060ti_v12_llama.py
--- POOR MAN'S PORTABLE FINETUNER v10 ---
Loading Tokenizer...
Loading dataset: B:\8B\!models--SicariusSicariiStuff--Llama-3.1-Nemotron-8B-UltraLong-1M-Instruct_Abliterated\dataset_cache\unified_dataset.parquet
Formatting dataset for Llama 3...
Map: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 665/665 [00:00<00:00, 41563.65 examples/s]
Training on 665 distinct Q&A pairs.
Loading Model with VRAM Augmentation (Limit: 4GiB)...
Loading Model with Manual CPU Offload (VRAM Cap: 4GiB)...
`torch_dtype` is deprecated! Use `dtype` instead!
Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:07<00:00,  1.02s/it]
Some parameters are on the meta device because they were offloaded to the cpu.
Applying surgical float32 casts to Norm layers...
Applying formatting function to train dataset: 100%|██████████████████████████████████████████████████████████████████████████████████████████████| 665/665 [00:00<00:00, 34994.63 examples/s]
Adding EOS to train dataset: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 665/665 [00:00<00:00, 39116.64 examples/s]
Tokenizing train dataset: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 665/665 [00:00<00:00, 2074.53 examples/s]
Truncating train dataset: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 665/665 [00:00<00:00, 332207.26 examples/s]
Starting Training...
The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'eos_token_id': 128009, 'pad_token_id': 128009}.
{'loss': 2.1798, 'grad_norm': 0.8218153119087219, 'learning_rate': 0.0, 'entropy': 1.5662660896778107, 'num_tokens': 2533.0, 'mean_token_accuracy': 0.5355911254882812, 'epoch': 0.01}
{'loss': 2.3164, 'grad_norm': 0.9427235126495361, 'learning_rate': 3.846153846153847e-06, 'entropy': 1.6257197856903076, 'num_tokens': 4558.0, 'mean_token_accuracy': 0.49528219550848007, 'epoch': 0.01}
{'loss': 2.1298, 'grad_norm': 1.0283432006835938, 'learning_rate': 7.692307692307694e-06, 'entropy': 1.4974118173122406, 'num_tokens': 6300.0, 'mean_token_accuracy': 0.5446272790431976, 'epoch': 0.02}
{'loss': 2.2291, 'grad_norm': 1.6822551488876343, 'learning_rate': 1.153846153846154e-05, 'entropy': 1.6818975508213043, 'num_tokens': 7779.0, 'mean_token_accuracy': 0.5187952369451523, 'epoch': 0.02}
{'loss': 2.411, 'grad_norm': 1.3989911079406738, 'learning_rate': 1.5384615384615387e-05, 'entropy': 1.7168107330799103, 'num_tokens': 9082.0, 'mean_token_accuracy': 0.510143868625164, 'epoch': 0.03}
{'loss': 2.3368, 'grad_norm': 1.6276675462722778, 'learning_rate': 1.923076923076923e-05, 'entropy': 1.7281462252140045, 'num_tokens': 10338.0, 'mean_token_accuracy': 0.5026054754853249, 'epoch': 0.04}
{'loss': 2.3965, 'grad_norm': 1.144151210784912, 'learning_rate': 2.307692307692308e-05, 'entropy': 1.7676715850830078, 'num_tokens': 11551.0, 'mean_token_accuracy': 0.5121526792645454, 'epoch': 0.04}
{'loss': 2.4239, 'grad_norm': 1.3351788520812988, 'learning_rate': 2.6923076923076923e-05, 'entropy': 1.6334916055202484, 'num_tokens': 12737.0, 'mean_token_accuracy': 0.4958110377192497, 'epoch': 0.05}
{'loss': 2.3206, 'grad_norm': 1.2621963024139404, 'learning_rate': 3.0769230769230774e-05, 'entropy': 1.7594745755195618, 'num_tokens': 13901.0, 'mean_token_accuracy': 0.5119918510317802, 'epoch': 0.05}
{'loss': 2.2701, 'grad_norm': 1.2281334400177002, 'learning_rate': 3.461538461538462e-05, 'entropy': 1.7398152649402618, 'num_tokens': 15031.0, 'mean_token_accuracy': 0.5213459506630898, 'epoch': 0.06}
{'loss': 2.4868, 'grad_norm': 1.4432495832443237, 'learning_rate': 3.846153846153846e-05, 'entropy': 1.9150506854057312, 'num_tokens': 16088.0, 'mean_token_accuracy': 0.4965682402253151, 'epoch': 0.07}
{'loss': 2.6584, 'grad_norm': 1.4506950378417969, 'learning_rate': 4.230769230769231e-05, 'entropy': 2.0016297101974487, 'num_tokens': 17100.0, 'mean_token_accuracy': 0.4782413989305496, 'epoch': 0.07}
{'loss': 2.421, 'grad_norm': 1.563546895980835, 'learning_rate': 4.615384615384616e-05, 'entropy': 2.005508542060852, 'num_tokens': 18080.0, 'mean_token_accuracy': 0.48652973026037216, 'epoch': 0.08}
{'loss': 2.2625, 'grad_norm': 2.7091758251190186, 'learning_rate': 5e-05, 'entropy': 1.9208467304706573, 'num_tokens': 19032.0, 'mean_token_accuracy': 0.5167630463838577, 'epoch': 0.08}
{'loss': 2.0083, 'grad_norm': 1.7311205863952637, 'learning_rate': 5.384615384615385e-05, 'entropy': 1.779516339302063, 'num_tokens': 19954.0, 'mean_token_accuracy': 0.573837161064148, 'epoch': 0.09}
{'loss': 2.2355, 'grad_norm': 2.44893479347229, 'learning_rate': 5.769230769230769e-05, 'entropy': 2.119834065437317, 'num_tokens': 20851.0, 'mean_token_accuracy': 0.5296452790498734, 'epoch': 0.1}
{'loss': 2.2688, 'grad_norm': 2.8306422233581543, 'learning_rate': 6.153846153846155e-05, 'entropy': 2.10884228348732, 'num_tokens': 21728.0, 'mean_token_accuracy': 0.5066874250769615, 'epoch': 0.1}
{'loss': 1.746, 'grad_norm': 2.7256522178649902, 'learning_rate': 6.538461538461539e-05, 'entropy': 1.711792677640915, 'num_tokens': 22569.0, 'mean_token_accuracy': 0.5976409167051315, 'epoch': 0.11}
{'loss': 2.5502, 'grad_norm': 1.5059473514556885, 'learning_rate': 6.923076923076924e-05, 'entropy': 2.5403071641921997, 'num_tokens': 23384.0, 'mean_token_accuracy': 0.5019567608833313, 'epoch': 0.11}
{'loss': 2.347, 'grad_norm': 1.2922959327697754, 'learning_rate': 7.307692307692307e-05, 'entropy': 2.550511956214905, 'num_tokens': 24181.0, 'mean_token_accuracy': 0.5208497494459152, 'epoch': 0.12}
{'loss': 2.018, 'grad_norm': 1.2528678178787231, 'learning_rate': 7.692307692307693e-05, 'entropy': 2.2230250239372253, 'num_tokens': 24965.0, 'mean_token_accuracy': 0.5665687024593353, 'epoch': 0.13}
{'loss': 2.1705, 'grad_norm': 1.3070764541625977, 'learning_rate': 8.076923076923078e-05, 'entropy': 2.247088372707367, 'num_tokens': 25730.0, 'mean_token_accuracy': 0.5373675227165222, 'epoch': 0.13}
{'loss': 2.2801, 'grad_norm': 1.8817178010940552, 'learning_rate': 8.461538461538461e-05, 'entropy': 2.5298433899879456, 'num_tokens': 26434.0, 'mean_token_accuracy': 0.5301968157291412, 'epoch': 0.14}
{'loss': 1.8557, 'grad_norm': 1.3850667476654053, 'learning_rate': 8.846153846153847e-05, 'entropy': 2.305492877960205, 'num_tokens': 27123.0, 'mean_token_accuracy': 0.5909797847270966, 'epoch': 0.14}
{'loss': 2.0643, 'grad_norm': 1.5376325845718384, 'learning_rate': 9.230769230769232e-05, 'entropy': 2.3065348863601685, 'num_tokens': 27792.0, 'mean_token_accuracy': 0.54416823387146, 'epoch': 0.15}
{'loss': 2.0294, 'grad_norm': 3.287940740585327, 'learning_rate': 9.615384615384617e-05, 'entropy': 2.3198046684265137, 'num_tokens': 28449.0, 'mean_token_accuracy': 0.5514661595225334, 'epoch': 0.16}
{'loss': 2.0958, 'grad_norm': 1.3331036567687988, 'learning_rate': 0.0001, 'entropy': 2.1669031977653503, 'num_tokens': 29066.0, 'mean_token_accuracy': 0.5613081604242325, 'epoch': 0.16}
{'loss': 1.9962, 'grad_norm': 1.6021634340286255, 'learning_rate': 9.999962299929265e-05, 'entropy': 2.1878325939178467, 'num_tokens': 29670.0, 'mean_token_accuracy': 0.5664236396551132, 'epoch': 0.17}
{'loss': 1.9289, 'grad_norm': 1.4902217388153076, 'learning_rate': 9.999849200285573e-05, 'entropy': 2.1001837253570557, 'num_tokens': 30259.0, 'mean_token_accuracy': 0.5893597304821014, 'epoch': 0.17}
{'loss': 2.2802, 'grad_norm': 1.756393313407898, 'learning_rate': 9.999660702774474e-05, 'entropy': 2.2907257676124573, 'num_tokens': 30821.0, 'mean_token_accuracy': 0.5180113017559052, 'epoch': 0.18}
{'loss': 1.8346, 'grad_norm': 1.50323486328125, 'learning_rate': 9.999396810238513e-05, 'entropy': 1.9585947096347809, 'num_tokens': 31367.0, 'mean_token_accuracy': 0.5811394453048706, 'epoch': 0.19}
{'loss': 1.949, 'grad_norm': 2.096813201904297, 'learning_rate': 9.999057526657196e-05, 'entropy': 2.109057664871216, 'num_tokens': 31884.0, 'mean_token_accuracy': 0.567564532160759, 'epoch': 0.19}
{'loss': 2.0046, 'grad_norm': 2.419137716293335, 'learning_rate': 9.998642857146934e-05, 'entropy': 2.0012708008289337, 'num_tokens': 32375.0, 'mean_token_accuracy': 0.6261561363935471, 'epoch': 0.2}
{'loss': 1.9485, 'grad_norm': 2.1153695583343506, 'learning_rate': 9.99815280796095e-05, 'entropy': 2.0116805136203766, 'num_tokens': 32854.0, 'mean_token_accuracy': 0.5809203386306763, 'epoch': 0.2}
{'loss': 1.7253, 'grad_norm': 1.9619979858398438, 'learning_rate': 9.997587386489202e-05, 'entropy': 1.8118351101875305, 'num_tokens': 33308.0, 'mean_token_accuracy': 0.591243177652359, 'epoch': 0.21}
{'loss': 1.6522, 'grad_norm': 2.4790496826171875, 'learning_rate': 9.996946601258261e-05, 'entropy': 1.660794198513031, 'num_tokens': 33731.0, 'mean_token_accuracy': 0.6089285761117935, 'epoch': 0.22}
{'loss': 1.616, 'grad_norm': 2.1125807762145996, 'learning_rate': 9.996230461931186e-05, 'entropy': 1.7168469727039337, 'num_tokens': 34138.0, 'mean_token_accuracy': 0.637673556804657, 'epoch': 0.22}
{'loss': 1.9039, 'grad_norm': 2.2153372764587402, 'learning_rate': 9.995438979307381e-05, 'entropy': 1.7653800249099731, 'num_tokens': 34493.0, 'mean_token_accuracy': 0.6017777472734451, 'epoch': 0.23}
{'loss': 2.1425, 'grad_norm': 2.5268499851226807, 'learning_rate': 9.994572165322422e-05, 'entropy': 2.01798939704895, 'num_tokens': 34832.0, 'mean_token_accuracy': 0.5281483083963394, 'epoch': 0.23}
{'loss': 1.9547, 'grad_norm': 4.172223091125488, 'learning_rate': 9.993630033047891e-05, 'entropy': 1.942413330078125, 'num_tokens': 35158.0, 'mean_token_accuracy': 0.54398974776268, 'epoch': 0.24}
{'loss': 1.8143, 'grad_norm': 3.2461745738983154, 'learning_rate': 9.992612596691171e-05, 'entropy': 2.0974113643169403, 'num_tokens': 35461.0, 'mean_token_accuracy': 0.6009708493947983, 'epoch': 0.25}
{'loss': 1.7534, 'grad_norm': 1.7432345151901245, 'learning_rate': 9.991519871595227e-05, 'entropy': 1.7236852645874023, 'num_tokens': 38032.0, 'mean_token_accuracy': 0.5768449306488037, 'epoch': 0.25}
{'loss': 1.6977, 'grad_norm': 1.6695091724395752, 'learning_rate': 9.990351874238388e-05, 'entropy': 1.6812101900577545, 'num_tokens': 40099.0, 'mean_token_accuracy': 0.5916101634502411, 'epoch': 0.26}
{'loss': 1.9002, 'grad_norm': 1.221008539199829, 'learning_rate': 9.989108622234084e-05, 'entropy': 1.853236347436905, 'num_tokens': 41971.0, 'mean_token_accuracy': 0.5501984655857086, 'epoch': 0.26}
{'loss': 1.6854, 'grad_norm': 1.1821712255477905, 'learning_rate': 9.987790134330593e-05, 'entropy': 1.6879638731479645, 'num_tokens': 43632.0, 'mean_token_accuracy': 0.5888950377702713, 'epoch': 0.27}
{'loss': 1.8243, 'grad_norm': 1.1733769178390503, 'learning_rate': 9.986396430410749e-05, 'entropy': 1.7590214014053345, 'num_tokens': 45042.0, 'mean_token_accuracy': 0.5628818422555923, 'epoch': 0.28}
{'loss': 1.8459, 'grad_norm': 1.057795763015747, 'learning_rate': 9.984927531491648e-05, 'entropy': 1.9597107470035553, 'num_tokens': 46296.0, 'mean_token_accuracy': 0.575367659330368, 'epoch': 0.28}
{'loss': 1.8242, 'grad_norm': 1.1011550426483154, 'learning_rate': 9.983383459724322e-05, 'entropy': 1.8689045011997223, 'num_tokens': 47522.0, 'mean_token_accuracy': 0.5769995152950287, 'epoch': 0.29}
{'loss': 1.7934, 'grad_norm': 0.9578100442886353, 'learning_rate': 9.981764238393424e-05, 'entropy': 1.8821882605552673, 'num_tokens': 48716.0, 'mean_token_accuracy': 0.5822672247886658, 'epoch': 0.29}
{'loss': 1.8623, 'grad_norm': 1.1741857528686523, 'learning_rate': 9.980069891916854e-05, 'entropy': 1.8619018197059631, 'num_tokens': 49869.0, 'mean_token_accuracy': 0.559070274233818, 'epoch': 0.3}
{'loss': 2.0382, 'grad_norm': 1.2271146774291992, 'learning_rate': 9.978300445845404e-05, 'entropy': 2.075949728488922, 'num_tokens': 50997.0, 'mean_token_accuracy': 0.5321046486496925, 'epoch': 0.31}
{'loss': 1.8374, 'grad_norm': 1.2754955291748047, 'learning_rate': 9.976455926862371e-05, 'entropy': 1.8476427793502808, 'num_tokens': 52076.0, 'mean_token_accuracy': 0.5663594752550125, 'epoch': 0.31}
{'loss': 2.1081, 'grad_norm': 1.7234376668930054, 'learning_rate': 9.974536362783156e-05, 'entropy': 2.003211945295334, 'num_tokens': 53131.0, 'mean_token_accuracy': 0.5271842181682587, 'epoch': 0.32}
{'loss': 1.5889, 'grad_norm': 2.359753131866455, 'learning_rate': 9.972541782554836e-05, 'entropy': 1.607701599597931, 'num_tokens': 54158.0, 'mean_token_accuracy': 0.630528561770916, 'epoch': 0.32}
{'loss': 1.8018, 'grad_norm': 1.1571866273880005, 'learning_rate': 9.970472216255741e-05, 'entropy': 1.8844026327133179, 'num_tokens': 55146.0, 'mean_token_accuracy': 0.5727912932634354, 'epoch': 0.33}
{'loss': 1.7964, 'grad_norm': 2.9389004707336426, 'learning_rate': 9.968327695094987e-05, 'entropy': 1.8064050674438477, 'num_tokens': 56111.0, 'mean_token_accuracy': 0.5640705525875092, 'epoch': 0.34}
{'loss': 1.8768, 'grad_norm': 1.1771553754806519, 'learning_rate': 9.966108251412014e-05, 'entropy': 2.0355775356292725, 'num_tokens': 57055.0, 'mean_token_accuracy': 0.578627273440361, 'epoch': 0.34}
{'loss': 1.8426, 'grad_norm': 1.8002325296401978, 'learning_rate': 9.963813918676095e-05, 'entropy': 1.864669382572174, 'num_tokens': 57974.0, 'mean_token_accuracy': 0.5800923705101013, 'epoch': 0.35}
{'loss': 1.8054, 'grad_norm': 1.495760440826416, 'learning_rate': 9.961444731485836e-05, 'entropy': 1.9267215430736542, 'num_tokens': 58868.0, 'mean_token_accuracy': 0.5741576850414276, 'epoch': 0.35}
{'loss': 1.4943, 'grad_norm': 2.09474515914917, 'learning_rate': 9.959000725568641e-05, 'entropy': 1.641057699918747, 'num_tokens': 59744.0, 'mean_token_accuracy': 0.6136951744556427, 'epoch': 0.36}
{'loss': 1.7523, 'grad_norm': 3.296443462371826, 'learning_rate': 9.956481937780193e-05, 'entropy': 1.8795695006847382, 'num_tokens': 60599.0, 'mean_token_accuracy': 0.5829629600048065, 'epoch': 0.37}
{'loss': 1.8354, 'grad_norm': 1.822119116783142, 'learning_rate': 9.953888406103883e-05, 'entropy': 1.9691583812236786, 'num_tokens': 61442.0, 'mean_token_accuracy': 0.6007373780012131, 'epoch': 0.37}
{'loss': 1.7909, 'grad_norm': 1.2213889360427856, 'learning_rate': 9.951220169650239e-05, 'entropy': 1.919170618057251, 'num_tokens': 62259.0, 'mean_token_accuracy': 0.5782864689826965, 'epoch': 0.38}
{'loss': 1.73, 'grad_norm': 1.713629126548767, 'learning_rate': 9.948477268656345e-05, 'entropy': 1.8828186690807343, 'num_tokens': 63053.0, 'mean_token_accuracy': 0.5810688585042953, 'epoch': 0.38}
{'loss': 1.9581, 'grad_norm': 2.117863655090332, 'learning_rate': 9.945659744485225e-05, 'entropy': 2.0405538082122803, 'num_tokens': 63812.0, 'mean_token_accuracy': 0.6026361435651779, 'epoch': 0.39}
{'loss': 1.9156, 'grad_norm': 2.013145923614502, 'learning_rate': 9.942767639625224e-05, 'entropy': 1.9021817743778229, 'num_tokens': 64540.0, 'mean_token_accuracy': 0.5591225624084473, 'epoch': 0.4}
{'loss': 2.1386, 'grad_norm': 1.5318644046783447, 'learning_rate': 9.939800997689362e-05, 'entropy': 2.0722250044345856, 'num_tokens': 65240.0, 'mean_token_accuracy': 0.5357324853539467, 'epoch': 0.4}
{'loss': 1.759, 'grad_norm': 1.8367432355880737, 'learning_rate': 9.936759863414685e-05, 'entropy': 1.8152096569538116, 'num_tokens': 65919.0, 'mean_token_accuracy': 0.5928448587656021, 'epoch': 0.41}
{'loss': 1.5746, 'grad_norm': 1.4440765380859375, 'learning_rate': 9.933644282661586e-05, 'entropy': 1.7293733358383179, 'num_tokens': 66578.0, 'mean_token_accuracy': 0.6093089431524277, 'epoch': 0.42}
{'loss': 1.8864, 'grad_norm': 1.6686040163040161, 'learning_rate': 9.930454302413108e-05, 'entropy': 1.8053934574127197, 'num_tokens': 67221.0, 'mean_token_accuracy': 0.5885429382324219, 'epoch': 0.42}
{'loss': 1.6931, 'grad_norm': 1.9901719093322754, 'learning_rate': 9.927189970774245e-05, 'entropy': 1.7563508450984955, 'num_tokens': 67850.0, 'mean_token_accuracy': 0.5872482806444168, 'epoch': 0.43}
{'loss': 1.7818, 'grad_norm': 1.9217439889907837, 'learning_rate': 9.923851336971208e-05, 'entropy': 1.802720457315445, 'num_tokens': 68462.0, 'mean_token_accuracy': 0.5886248648166656, 'epoch': 0.43}
{'loss': 1.7137, 'grad_norm': 1.6363177299499512, 'learning_rate': 9.920438451350694e-05, 'entropy': 1.7130776345729828, 'num_tokens': 69055.0, 'mean_token_accuracy': 0.6049788743257523, 'epoch': 0.44}
{'loss': 1.9381, 'grad_norm': 1.6792877912521362, 'learning_rate': 9.916951365379112e-05, 'entropy': 1.94430872797966, 'num_tokens': 69625.0, 'mean_token_accuracy': 0.5508886352181435, 'epoch': 0.45}
{'loss': 1.7549, 'grad_norm': 1.7125887870788574, 'learning_rate': 9.913390131641815e-05, 'entropy': 1.8223829865455627, 'num_tokens': 70179.0, 'mean_token_accuracy': 0.5892161577939987, 'epoch': 0.45}
{'loss': 1.7903, 'grad_norm': 1.692474603652954, 'learning_rate': 9.909754803842313e-05, 'entropy': 1.7727929949760437, 'num_tokens': 70710.0, 'mean_token_accuracy': 0.5885491520166397, 'epoch': 0.46}
{'loss': 1.7135, 'grad_norm': 1.8748955726623535, 'learning_rate': 9.906045436801448e-05, 'entropy': 1.7988978326320648, 'num_tokens': 71225.0, 'mean_token_accuracy': 0.6178579777479172, 'epoch': 0.46}
{'loss': 1.7836, 'grad_norm': 1.7148672342300415, 'learning_rate': 9.902262086456582e-05, 'entropy': 1.8063230216503143, 'num_tokens': 71716.0, 'mean_token_accuracy': 0.5989696532487869, 'epoch': 0.47}
{'loss': 1.7086, 'grad_norm': 1.7662420272827148, 'learning_rate': 9.898404809860744e-05, 'entropy': 1.8168116807937622, 'num_tokens': 72156.0, 'mean_token_accuracy': 0.6334411203861237, 'epoch': 0.48}
{'loss': 1.803, 'grad_norm': 2.0751891136169434, 'learning_rate': 9.894473665181776e-05, 'entropy': 1.695672333240509, 'num_tokens': 72539.0, 'mean_token_accuracy': 0.600599855184555, 'epoch': 0.48}
{'loss': 1.5374, 'grad_norm': 2.183985710144043, 'learning_rate': 9.89046871170145e-05, 'entropy': 1.730334460735321, 'num_tokens': 72881.0, 'mean_token_accuracy': 0.5923311114311218, 'epoch': 0.49}
{'loss': 1.8063, 'grad_norm': 2.259176254272461, 'learning_rate': 9.886390009814579e-05, 'entropy': 1.850230187177658, 'num_tokens': 73165.0, 'mean_token_accuracy': 0.5854910463094711, 'epoch': 0.49}
{'loss': 1.5453, 'grad_norm': 1.2147005796432495, 'learning_rate': 9.882237621028101e-05, 'entropy': 1.5008636116981506, 'num_tokens': 75201.0, 'mean_token_accuracy': 0.6172028332948685, 'epoch': 0.5}
{'loss': 1.8514, 'grad_norm': 1.4409785270690918, 'learning_rate': 9.878011607960156e-05, 'entropy': 1.7158764004707336, 'num_tokens': 76915.0, 'mean_token_accuracy': 0.5538609102368355, 'epoch': 0.51}
{'loss': 1.6749, 'grad_norm': 1.753063440322876, 'learning_rate': 9.873712034339143e-05, 'entropy': 1.6282544434070587, 'num_tokens': 78464.0, 'mean_token_accuracy': 0.6052332669496536, 'epoch': 0.51}
{'loss': 1.6184, 'grad_norm': 1.080826759338379, 'learning_rate': 9.869338965002752e-05, 'entropy': 1.6545091569423676, 'num_tokens': 79852.0, 'mean_token_accuracy': 0.6016214936971664, 'epoch': 0.52}
{'loss': 1.5405, 'grad_norm': 1.1354306936264038, 'learning_rate': 9.864892465896994e-05, 'entropy': 1.58961820602417, 'num_tokens': 81191.0, 'mean_token_accuracy': 0.6119740456342697, 'epoch': 0.52}
{'loss': 1.469, 'grad_norm': 1.0038763284683228, 'learning_rate': 9.860372604075199e-05, 'entropy': 1.6042021811008453, 'num_tokens': 82499.0, 'mean_token_accuracy': 0.6473474353551865, 'epoch': 0.53}
{'loss': 1.3556, 'grad_norm': 1.0039583444595337, 'learning_rate': 9.855779447697013e-05, 'entropy': 1.4880214929580688, 'num_tokens': 83773.0, 'mean_token_accuracy': 0.6497301906347275, 'epoch': 0.54}
{'loss': 1.5756, 'grad_norm': 1.0446619987487793, 'learning_rate': 9.851113066027364e-05, 'entropy': 1.6069844961166382, 'num_tokens': 84946.0, 'mean_token_accuracy': 0.6029240041971207, 'epoch': 0.54}
{'loss': 1.4589, 'grad_norm': 1.1369856595993042, 'learning_rate': 9.846373529435418e-05, 'entropy': 1.5924058556556702, 'num_tokens': 86073.0, 'mean_token_accuracy': 0.6359071284532547, 'epoch': 0.55}
{'loss': 1.7776, 'grad_norm': 1.656422734260559, 'learning_rate': 9.841560909393523e-05, 'entropy': 1.7444784343242645, 'num_tokens': 87143.0, 'mean_token_accuracy': 0.5560315698385239, 'epoch': 0.55}
{'loss': 1.4717, 'grad_norm': 1.137046456336975, 'learning_rate': 9.836675278476124e-05, 'entropy': 1.6254683136940002, 'num_tokens': 88186.0, 'mean_token_accuracy': 0.6085537225008011, 'epoch': 0.56}
{'loss': 1.8612, 'grad_norm': 1.6095246076583862, 'learning_rate': 9.831716710358673e-05, 'entropy': 1.9070601165294647, 'num_tokens': 89189.0, 'mean_token_accuracy': 0.5879168957471848, 'epoch': 0.57}
{'loss': 1.8497, 'grad_norm': 2.342237710952759, 'learning_rate': 9.82668527981652e-05, 'entropy': 1.7799060344696045, 'num_tokens': 90146.0, 'mean_token_accuracy': 0.5812483727931976, 'epoch': 0.57}
{'loss': 1.8291, 'grad_norm': 1.3020350933074951, 'learning_rate': 9.821581062723779e-05, 'entropy': 1.7243834733963013, 'num_tokens': 91074.0, 'mean_token_accuracy': 0.5943600684404373, 'epoch': 0.58}
{'loss': 1.6448, 'grad_norm': 1.9539293050765991, 'learning_rate': 9.816404136052186e-05, 'entropy': 1.5819264650344849, 'num_tokens': 91975.0, 'mean_token_accuracy': 0.6075610518455505, 'epoch': 0.58}
{'loss': 1.8533, 'grad_norm': 1.4489495754241943, 'learning_rate': 9.811154577869943e-05, 'entropy': 1.715063065290451, 'num_tokens': 92855.0, 'mean_token_accuracy': 0.5879998952150345, 'epoch': 0.59}
{'loss': 1.6998, 'grad_norm': 1.6818108558654785, 'learning_rate': 9.805832467340538e-05, 'entropy': 1.7500436902046204, 'num_tokens': 93714.0, 'mean_token_accuracy': 0.6024422645568848, 'epoch': 0.6}
{'loss': 1.6656, 'grad_norm': 1.9722617864608765, 'learning_rate': 9.800437884721545e-05, 'entropy': 1.6412696540355682, 'num_tokens': 94557.0, 'mean_token_accuracy': 0.6047296524047852, 'epoch': 0.6}
{'loss': 1.7864, 'grad_norm': 1.3377068042755127, 'learning_rate': 9.794970911363426e-05, 'entropy': 1.7491322755813599, 'num_tokens': 95362.0, 'mean_token_accuracy': 0.5815430581569672, 'epoch': 0.61}
{'loss': 1.6923, 'grad_norm': 1.5603810548782349, 'learning_rate': 9.78943162970829e-05, 'entropy': 1.7641785740852356, 'num_tokens': 96152.0, 'mean_token_accuracy': 0.6285222172737122, 'epoch': 0.61}
{'loss': 1.7955, 'grad_norm': 2.1838274002075195, 'learning_rate': 9.783820123288664e-05, 'entropy': 1.8075791895389557, 'num_tokens': 96926.0, 'mean_token_accuracy': 0.5611911565065384, 'epoch': 0.62}
{'loss': 1.9006, 'grad_norm': 1.9511891603469849, 'learning_rate': 9.778136476726223e-05, 'entropy': 1.8083418309688568, 'num_tokens': 97689.0, 'mean_token_accuracy': 0.5773636475205421, 'epoch': 0.63}
{'loss': 2.0231, 'grad_norm': 2.0574676990509033, 'learning_rate': 9.772380775730516e-05, 'entropy': 1.933795005083084, 'num_tokens': 98432.0, 'mean_token_accuracy': 0.5467464849352837, 'epoch': 0.63}
{'loss': 1.8158, 'grad_norm': 1.9754457473754883, 'learning_rate': 9.766553107097681e-05, 'entropy': 1.8470250070095062, 'num_tokens': 99158.0, 'mean_token_accuracy': 0.6007344275712967, 'epoch': 0.64}
{'loss': 2.0242, 'grad_norm': 2.1607322692871094, 'learning_rate': 9.760653558709122e-05, 'entropy': 1.9772748947143555, 'num_tokens': 99857.0, 'mean_token_accuracy': 0.5825072675943375, 'epoch': 0.64}
{'loss': 1.6285, 'grad_norm': 1.9444059133529663, 'learning_rate': 9.754682219530199e-05, 'entropy': 1.8473709225654602, 'num_tokens': 100541.0, 'mean_token_accuracy': 0.608946219086647, 'epoch': 0.65}
{'loss': 1.575, 'grad_norm': 1.582433819770813, 'learning_rate': 9.748639179608872e-05, 'entropy': 1.728728711605072, 'num_tokens': 101218.0, 'mean_token_accuracy': 0.6198175400495529, 'epoch': 0.66}
{'loss': 1.7044, 'grad_norm': 1.7135034799575806, 'learning_rate': 9.742524530074355e-05, 'entropy': 1.7502144277095795, 'num_tokens': 101875.0, 'mean_token_accuracy': 0.6010355204343796, 'epoch': 0.66}
{'loss': 1.8239, 'grad_norm': 1.7812259197235107, 'learning_rate': 9.736338363135738e-05, 'entropy': 1.682581603527069, 'num_tokens': 102491.0, 'mean_token_accuracy': 0.5735767930746078, 'epoch': 0.67}
{'loss': 1.7118, 'grad_norm': 1.5981183052062988, 'learning_rate': 9.730080772080592e-05, 'entropy': 1.761479914188385, 'num_tokens': 103095.0, 'mean_token_accuracy': 0.6183333396911621, 'epoch': 0.67}
{'loss': 1.8709, 'grad_norm': 1.4970611333847046, 'learning_rate': 9.723751851273568e-05, 'entropy': 1.9604682326316833, 'num_tokens': 103692.0, 'mean_token_accuracy': 0.5904713124036789, 'epoch': 0.68}
{'loss': 1.6963, 'grad_norm': 1.6478865146636963, 'learning_rate': 9.71735169615497e-05, 'entropy': 1.8344905078411102, 'num_tokens': 104279.0, 'mean_token_accuracy': 0.6311911344528198, 'epoch': 0.69}
{'loss': 1.6128, 'grad_norm': 1.7165558338165283, 'learning_rate': 9.710880403239317e-05, 'entropy': 1.7161068618297577, 'num_tokens': 104843.0, 'mean_token_accuracy': 0.6144774109125137, 'epoch': 0.69}
{'loss': 1.4177, 'grad_norm': 1.7181566953659058, 'learning_rate': 9.704338070113893e-05, 'entropy': 1.7200258672237396, 'num_tokens': 105381.0, 'mean_token_accuracy': 0.6502786427736282, 'epoch': 0.7}
{'loss': 1.7275, 'grad_norm': 1.685890793800354, 'learning_rate': 9.697724795437264e-05, 'entropy': 1.7534075677394867, 'num_tokens': 105889.0, 'mean_token_accuracy': 0.6505307257175446, 'epoch': 0.7}
{'loss': 1.7642, 'grad_norm': 1.771565318107605, 'learning_rate': 9.6910406789378e-05, 'entropy': 1.7137005627155304, 'num_tokens': 106365.0, 'mean_token_accuracy': 0.5891141965985298, 'epoch': 0.71}
{'loss': 1.648, 'grad_norm': 1.7961786985397339, 'learning_rate': 9.684285821412165e-05, 'entropy': 1.7436869740486145, 'num_tokens': 106820.0, 'mean_token_accuracy': 0.6271873116493225, 'epoch': 0.72}
{'loss': 1.7777, 'grad_norm': 2.677800178527832, 'learning_rate': 9.677460324723806e-05, 'entropy': 1.8210408091545105, 'num_tokens': 107258.0, 'mean_token_accuracy': 0.645370364189148, 'epoch': 0.72}
{'loss': 1.7291, 'grad_norm': 1.971798062324524, 'learning_rate': 9.670564291801401e-05, 'entropy': 1.707783967256546, 'num_tokens': 107671.0, 'mean_token_accuracy': 0.6390751600265503, 'epoch': 0.73}
{'loss': 1.6259, 'grad_norm': 1.8861751556396484, 'learning_rate': 9.663597826637325e-05, 'entropy': 1.6464769840240479, 'num_tokens': 108020.0, 'mean_token_accuracy': 0.5924588590860367, 'epoch': 0.73}
{'loss': 1.7956, 'grad_norm': 2.095845937728882, 'learning_rate': 9.656561034286069e-05, 'entropy': 1.6791136264801025, 'num_tokens': 108315.0, 'mean_token_accuracy': 0.5838335305452347, 'epoch': 0.74}
{'loss': 1.5315, 'grad_norm': 0.8943172097206116, 'learning_rate': 9.64945402086266e-05, 'entropy': 1.4076267778873444, 'num_tokens': 111229.0, 'mean_token_accuracy': 0.6153527945280075, 'epoch': 0.75}
{'loss': 1.7993, 'grad_norm': 1.0444881916046143, 'learning_rate': 9.642276893541063e-05, 'entropy': 1.6091211438179016, 'num_tokens': 113140.0, 'mean_token_accuracy': 0.57386215031147, 'epoch': 0.75}
{'loss': 1.3987, 'grad_norm': 0.9943051934242249, 'learning_rate': 9.63502976055256e-05, 'entropy': 1.4389461278915405, 'num_tokens': 114811.0, 'mean_token_accuracy': 0.6361788958311081, 'epoch': 0.76}
{'loss': 1.5004, 'grad_norm': 0.897528350353241, 'learning_rate': 9.627712731184122e-05, 'entropy': 1.513084501028061, 'num_tokens': 116366.0, 'mean_token_accuracy': 0.5996393412351608, 'epoch': 0.76}
{'loss': 1.5569, 'grad_norm': 0.9425682425498962, 'learning_rate': 9.620325915776758e-05, 'entropy': 1.6013512015342712, 'num_tokens': 117832.0, 'mean_token_accuracy': 0.6141247749328613, 'epoch': 0.77}
{'loss': 1.6931, 'grad_norm': 1.0720947980880737, 'learning_rate': 9.612869425723854e-05, 'entropy': 1.6826394200325012, 'num_tokens': 119248.0, 'mean_token_accuracy': 0.5909788608551025, 'epoch': 0.78}
{'loss': 1.5612, 'grad_norm': 1.0194380283355713, 'learning_rate': 9.605343373469491e-05, 'entropy': 1.614288330078125, 'num_tokens': 120585.0, 'mean_token_accuracy': 0.6112469136714935, 'epoch': 0.78}
{'loss': 1.2966, 'grad_norm': 1.3797557353973389, 'learning_rate': 9.59774787250675e-05, 'entropy': 1.4267548620700836, 'num_tokens': 121877.0, 'mean_token_accuracy': 0.6646803170442581, 'epoch': 0.79}
{'loss': 1.6672, 'grad_norm': 1.0895981788635254, 'learning_rate': 9.590083037376001e-05, 'entropy': 1.719111293554306, 'num_tokens': 123130.0, 'mean_token_accuracy': 0.5974779278039932, 'epoch': 0.79}
{'loss': 1.6638, 'grad_norm': 1.0925134420394897, 'learning_rate': 9.582348983663173e-05, 'entropy': 1.698411613702774, 'num_tokens': 124308.0, 'mean_token_accuracy': 0.6041029542684555, 'epoch': 0.8}
{'loss': 1.7263, 'grad_norm': 1.2006117105484009, 'learning_rate': 9.574545827998017e-05, 'entropy': 1.848366230726242, 'num_tokens': 125420.0, 'mean_token_accuracy': 0.5926222652196884, 'epoch': 0.81}
{'loss': 1.88, 'grad_norm': 1.4124488830566406, 'learning_rate': 9.566673688052339e-05, 'entropy': 1.903512567281723, 'num_tokens': 126487.0, 'mean_token_accuracy': 0.5549195408821106, 'epoch': 0.81}
{'loss': 1.7306, 'grad_norm': 1.18662428855896, 'learning_rate': 9.558732682538233e-05, 'entropy': 1.769284427165985, 'num_tokens': 127530.0, 'mean_token_accuracy': 0.577408567070961, 'epoch': 0.82}
{'loss': 1.5439, 'grad_norm': 1.18385910987854, 'learning_rate': 9.550722931206286e-05, 'entropy': 1.6756306290626526, 'num_tokens': 128532.0, 'mean_token_accuracy': 0.623486801981926, 'epoch': 0.82}
{'loss': 1.8911, 'grad_norm': 2.3145337104797363, 'learning_rate': 9.542644554843778e-05, 'entropy': 1.8834153413772583, 'num_tokens': 129495.0, 'mean_token_accuracy': 0.5666824504733086, 'epoch': 0.83}
{'loss': 1.6338, 'grad_norm': 1.3079346418380737, 'learning_rate': 9.53449767527285e-05, 'entropy': 1.673485666513443, 'num_tokens': 130426.0, 'mean_token_accuracy': 0.6162396371364594, 'epoch': 0.84}
{'loss': 1.5454, 'grad_norm': 1.778074860572815, 'learning_rate': 9.526282415348677e-05, 'entropy': 1.6326853930950165, 'num_tokens': 131306.0, 'mean_token_accuracy': 0.6372188627719879, 'epoch': 0.84}
{'loss': 1.6839, 'grad_norm': 1.8547321557998657, 'learning_rate': 9.517998898957611e-05, 'entropy': 1.671204000711441, 'num_tokens': 132152.0, 'mean_token_accuracy': 0.6033399105072021, 'epoch': 0.85}
{'loss': 1.852, 'grad_norm': 1.7888838052749634, 'learning_rate': 9.509647251015314e-05, 'entropy': 1.8267798125743866, 'num_tokens': 132976.0, 'mean_token_accuracy': 0.5791603177785873, 'epoch': 0.85}
{'loss': 1.4806, 'grad_norm': 1.2037335634231567, 'learning_rate': 9.501227597464875e-05, 'entropy': 1.5865033864974976, 'num_tokens': 133783.0, 'mean_token_accuracy': 0.6475914865732193, 'epoch': 0.86}
{'loss': 1.7597, 'grad_norm': 1.985162377357483, 'learning_rate': 9.492740065274904e-05, 'entropy': 1.8017926812171936, 'num_tokens': 134525.0, 'mean_token_accuracy': 0.582187682390213, 'epoch': 0.87}
{'loss': 1.4539, 'grad_norm': 1.3150838613510132, 'learning_rate': 9.484184782437628e-05, 'entropy': 1.5580840408802032, 'num_tokens': 135232.0, 'mean_token_accuracy': 0.6207011044025421, 'epoch': 0.87}
{'loss': 1.8926, 'grad_norm': 1.5408499240875244, 'learning_rate': 9.475561877966956e-05, 'entropy': 1.7677281498908997, 'num_tokens': 135899.0, 'mean_token_accuracy': 0.5918874591588974, 'epoch': 0.88}
{'loss': 1.6324, 'grad_norm': 1.5907090902328491, 'learning_rate': 9.46687148189653e-05, 'entropy': 1.6555911600589752, 'num_tokens': 136545.0, 'mean_token_accuracy': 0.5887422263622284, 'epoch': 0.88}
{'loss': 1.827, 'grad_norm': 1.601693034172058, 'learning_rate': 9.458113725277769e-05, 'entropy': 1.7514843940734863, 'num_tokens': 137184.0, 'mean_token_accuracy': 0.5717001110315323, 'epoch': 0.89}
{'loss': 1.6872, 'grad_norm': 1.7727317810058594, 'learning_rate': 9.44928874017789e-05, 'entropy': 1.674159288406372, 'num_tokens': 137809.0, 'mean_token_accuracy': 0.6054814755916595, 'epoch': 0.9}
{'loss': 1.7399, 'grad_norm': 1.831101417541504, 'learning_rate': 9.440396659677919e-05, 'entropy': 1.8016743957996368, 'num_tokens': 138421.0, 'mean_token_accuracy': 0.5807951092720032, 'epoch': 0.9}
{'loss': 1.7636, 'grad_norm': 1.8827821016311646, 'learning_rate': 9.43143761787068e-05, 'entropy': 1.7538206577301025, 'num_tokens': 139019.0, 'mean_token_accuracy': 0.6027028262615204, 'epoch': 0.91}
{'loss': 1.7038, 'grad_norm': 1.8115235567092896, 'learning_rate': 9.422411749858779e-05, 'entropy': 1.6334579586982727, 'num_tokens': 139596.0, 'mean_token_accuracy': 0.6197989284992218, 'epoch': 0.91}
{'loss': 1.8862, 'grad_norm': 1.9638922214508057, 'learning_rate': 9.413319191752559e-05, 'entropy': 1.964978665113449, 'num_tokens': 140160.0, 'mean_token_accuracy': 0.5928977578878403, 'epoch': 0.92}
{'loss': 1.8659, 'grad_norm': 1.814119815826416, 'learning_rate': 9.404160080668055e-05, 'entropy': 1.7915958762168884, 'num_tokens': 140714.0, 'mean_token_accuracy': 0.5928854793310165, 'epoch': 0.93}
{'loss': 1.7412, 'grad_norm': 1.8771370649337769, 'learning_rate': 9.394934554724921e-05, 'entropy': 1.6734224259853363, 'num_tokens': 141256.0, 'mean_token_accuracy': 0.5890888273715973, 'epoch': 0.93}
{'loss': 1.6039, 'grad_norm': 1.7488360404968262, 'learning_rate': 9.385642753044348e-05, 'entropy': 1.7917393743991852, 'num_tokens': 141783.0, 'mean_token_accuracy': 0.6290485858917236, 'epoch': 0.94}
{'loss': 1.5773, 'grad_norm': 1.8186588287353516, 'learning_rate': 9.37628481574697e-05, 'entropy': 1.6509276032447815, 'num_tokens': 142291.0, 'mean_token_accuracy': 0.6228395402431488, 'epoch': 0.94}
{'loss': 1.7013, 'grad_norm': 2.092195987701416, 'learning_rate': 9.366860883950745e-05, 'entropy': 1.910431444644928, 'num_tokens': 142779.0, 'mean_token_accuracy': 0.6241789609193802, 'epoch': 0.95}
{'loss': 1.8086, 'grad_norm': 2.0366580486297607, 'learning_rate': 9.357371099768833e-05, 'entropy': 1.856508195400238, 'num_tokens': 143247.0, 'mean_token_accuracy': 0.5627056583762169, 'epoch': 0.96}
{'loss': 1.5578, 'grad_norm': 2.069629669189453, 'learning_rate': 9.347815606307445e-05, 'entropy': 1.676551729440689, 'num_tokens': 143703.0, 'mean_token_accuracy': 0.6351692378520966, 'epoch': 0.96}
{'loss': 1.5131, 'grad_norm': 1.8559539318084717, 'learning_rate': 9.338194547663694e-05, 'entropy': 1.6121012270450592, 'num_tokens': 144137.0, 'mean_token_accuracy': 0.6278430819511414, 'epoch': 0.97}
{'loss': 1.5646, 'grad_norm': 2.0773208141326904, 'learning_rate': 9.328508068923418e-05, 'entropy': 1.6353590488433838, 'num_tokens': 144561.0, 'mean_token_accuracy': 0.6479222923517227, 'epoch': 0.97}
{'loss': 1.5279, 'grad_norm': 1.9933832883834839, 'learning_rate': 9.31875631615899e-05, 'entropy': 1.5706246495246887, 'num_tokens': 144962.0, 'mean_token_accuracy': 0.627127394080162, 'epoch': 0.98}
{'loss': 1.6576, 'grad_norm': 2.37951397895813, 'learning_rate': 9.308939436427117e-05, 'entropy': 1.7116059064865112, 'num_tokens': 145264.0, 'mean_token_accuracy': 0.5851102769374847, 'epoch': 0.99}
{'loss': 1.7977, 'grad_norm': 1.4900258779525757, 'learning_rate': 9.299057577766622e-05, 'entropy': 1.8347175419330597, 'num_tokens': 146383.0, 'mean_token_accuracy': 0.5632264614105225, 'epoch': 0.99}
{'loss': 1.5567, 'grad_norm': 1.668030858039856, 'learning_rate': 9.289110889196214e-05, 'entropy': 1.6020879745483398, 'num_tokens': 147123.0, 'mean_token_accuracy': 0.6458230763673782, 'epoch': 1.0}
{'loss': 1.6373, 'grad_norm': 3.52628493309021, 'learning_rate': 9.279099520712238e-05, 'entropy': 1.5493701696395874, 'num_tokens': 147235.0, 'mean_token_accuracy': 0.630630612373352, 'epoch': 1.0}
{'loss': 1.4987, 'grad_norm': 0.8189054727554321, 'learning_rate': 9.269023623286417e-05, 'entropy': 1.4848939776420593, 'num_tokens': 149883.0, 'mean_token_accuracy': 0.6169576048851013, 'epoch': 1.01}
{'loss': 1.5418, 'grad_norm': 0.9761294722557068, 'learning_rate': 9.258883348863566e-05, 'entropy': 1.5730266869068146, 'num_tokens': 151794.0, 'mean_token_accuracy': 0.6159390658140182, 'epoch': 1.01}
{'loss': 1.3054, 'grad_norm': 0.9190660715103149, 'learning_rate': 9.248678850359309e-05, 'entropy': 1.400682657957077, 'num_tokens': 153442.0, 'mean_token_accuracy': 0.658955529332161, 'epoch': 1.02}
{'loss': 1.4421, 'grad_norm': 0.9182265996932983, 'learning_rate': 9.238410281657775e-05, 'entropy': 1.5661883354187012, 'num_tokens': 154960.0, 'mean_token_accuracy': 0.6336454898118973, 'epoch': 1.02}
{'loss': 1.2833, 'grad_norm': 0.9987655282020569, 'learning_rate': 9.228077797609269e-05, 'entropy': 1.4888449013233185, 'num_tokens': 156402.0, 'mean_token_accuracy': 0.6717199832201004, 'epoch': 1.03}
{'loss': 1.3613, 'grad_norm': 0.9975628852844238, 'learning_rate': 9.217681554027945e-05, 'entropy': 1.5470351576805115, 'num_tokens': 157765.0, 'mean_token_accuracy': 0.6483812928199768, 'epoch': 1.04}
{'loss': 1.5411, 'grad_norm': 1.1530799865722656, 'learning_rate': 9.207221707689447e-05, 'entropy': 1.519189715385437, 'num_tokens': 159023.0, 'mean_token_accuracy': 0.6290767937898636, 'epoch': 1.04}
{'loss': 1.402, 'grad_norm': 1.2022697925567627, 'learning_rate': 9.196698416328557e-05, 'entropy': 1.4957760870456696, 'num_tokens': 160227.0, 'mean_token_accuracy': 0.6472848951816559, 'epoch': 1.05}
{'loss': 1.3196, 'grad_norm': 1.2221548557281494, 'learning_rate': 9.186111838636804e-05, 'entropy': 1.4984432756900787, 'num_tokens': 161338.0, 'mean_token_accuracy': 0.6546680182218552, 'epoch': 1.05}
{'loss': 1.2538, 'grad_norm': 2.0080103874206543, 'learning_rate': 9.175462134260083e-05, 'entropy': 1.4374139606952667, 'num_tokens': 162395.0, 'mean_token_accuracy': 0.6773329377174377, 'epoch': 1.06}
{'loss': 0.9435, 'grad_norm': 1.3251744508743286, 'learning_rate': 9.16474946379623e-05, 'entropy': 1.1507239788770676, 'num_tokens': 163424.0, 'mean_token_accuracy': 0.743471309542656, 'epoch': 1.07}
{'loss': 1.4544, 'grad_norm': 2.1070516109466553, 'learning_rate': 9.153973988792626e-05, 'entropy': 1.4159432351589203, 'num_tokens': 164422.0, 'mean_token_accuracy': 0.6450655609369278, 'epoch': 1.07}
{'loss': 1.4867, 'grad_norm': 1.8195301294326782, 'learning_rate': 9.143135871743736e-05, 'entropy': 1.4863049983978271, 'num_tokens': 165389.0, 'mean_token_accuracy': 0.5866322219371796, 'epoch': 1.08}
{'loss': 1.149, 'grad_norm': 2.1307456493377686, 'learning_rate': 9.132235276088671e-05, 'entropy': 1.1223405599594116, 'num_tokens': 166344.0, 'mean_token_accuracy': 0.7331058084964752, 'epoch': 1.08}
{'loss': 1.2813, 'grad_norm': 1.7488305568695068, 'learning_rate': 9.121272366208722e-05, 'entropy': 1.2721976265311241, 'num_tokens': 167277.0, 'mean_token_accuracy': 0.7100204527378082, 'epoch': 1.09}
{'loss': 1.4286, 'grad_norm': 2.1057167053222656, 'learning_rate': 9.110247307424884e-05, 'entropy': 1.3949885964393616, 'num_tokens': 168182.0, 'mean_token_accuracy': 0.6547578573226929, 'epoch': 1.1}
{'loss': 1.439, 'grad_norm': 2.233682870864868, 'learning_rate': 9.09916026599535e-05, 'entropy': 1.365087866783142, 'num_tokens': 169074.0, 'mean_token_accuracy': 0.6362612545490265, 'epoch': 1.1}
{'loss': 1.154, 'grad_norm': 1.6215680837631226, 'learning_rate': 9.08801140911302e-05, 'entropy': 1.08425685018301, 'num_tokens': 169958.0, 'mean_token_accuracy': 0.702637106180191, 'epoch': 1.11}
{'loss': 1.1728, 'grad_norm': 1.6601067781448364, 'learning_rate': 9.076800904902975e-05, 'entropy': 1.1473501324653625, 'num_tokens': 170813.0, 'mean_token_accuracy': 0.7205572575330734, 'epoch': 1.11}
{'loss': 1.3875, 'grad_norm': 1.9390069246292114, 'learning_rate': 9.06552892241993e-05, 'entropy': 1.39456307888031, 'num_tokens': 171650.0, 'mean_token_accuracy': 0.6788348406553268, 'epoch': 1.12}
{'loss': 1.3808, 'grad_norm': 1.819263219833374, 'learning_rate': 9.054195631645704e-05, 'entropy': 1.4643152952194214, 'num_tokens': 172457.0, 'mean_token_accuracy': 0.6815787702798843, 'epoch': 1.13}
{'loss': 1.4031, 'grad_norm': 1.7229253053665161, 'learning_rate': 9.042801203486641e-05, 'entropy': 1.468374103307724, 'num_tokens': 173243.0, 'mean_token_accuracy': 0.6877857744693756, 'epoch': 1.13}
{'loss': 1.3799, 'grad_norm': 1.7230793237686157, 'learning_rate': 9.03134580977104e-05, 'entropy': 1.4747581481933594, 'num_tokens': 173990.0, 'mean_token_accuracy': 0.6579198986291885, 'epoch': 1.14}
{'loss': 1.1981, 'grad_norm': 2.0874085426330566, 'learning_rate': 9.019829623246563e-05, 'entropy': 1.3913747072219849, 'num_tokens': 174720.0, 'mean_token_accuracy': 0.7086922228336334, 'epoch': 1.14}
{'loss': 1.4502, 'grad_norm': 1.7336602210998535, 'learning_rate': 9.008252817577628e-05, 'entropy': 1.5993949174880981, 'num_tokens': 175418.0, 'mean_token_accuracy': 0.6671039313077927, 'epoch': 1.15}
{'loss': 1.2244, 'grad_norm': 1.9196568727493286, 'learning_rate': 8.99661556734279e-05, 'entropy': 1.4973070919513702, 'num_tokens': 176099.0, 'mean_token_accuracy': 0.6822766959667206, 'epoch': 1.16}
{'loss': 1.3523, 'grad_norm': 1.854437232017517, 'learning_rate': 8.984918048032116e-05, 'entropy': 1.4049434661865234, 'num_tokens': 176751.0, 'mean_token_accuracy': 0.6629657447338104, 'epoch': 1.16}
{'loss': 1.4486, 'grad_norm': 1.9118467569351196, 'learning_rate': 8.973160436044526e-05, 'entropy': 1.5358704924583435, 'num_tokens': 177382.0, 'mean_token_accuracy': 0.6454864144325256, 'epoch': 1.17}
{'loss': 1.3032, 'grad_norm': 2.1970303058624268, 'learning_rate': 8.961342908685142e-05, 'entropy': 1.482747197151184, 'num_tokens': 177989.0, 'mean_token_accuracy': 0.6535460948944092, 'epoch': 1.17}
{'loss': 1.3763, 'grad_norm': 2.0197017192840576, 'learning_rate': 8.949465644162611e-05, 'entropy': 1.4684979319572449, 'num_tokens': 178590.0, 'mean_token_accuracy': 0.6631767302751541, 'epoch': 1.18}
{'loss': 1.1329, 'grad_norm': 1.9651243686676025, 'learning_rate': 8.937528821586416e-05, 'entropy': 1.3571238815784454, 'num_tokens': 179182.0, 'mean_token_accuracy': 0.707035556435585, 'epoch': 1.19}
{'loss': 1.247, 'grad_norm': 2.059915542602539, 'learning_rate': 8.92553262096418e-05, 'entropy': 1.4267708361148834, 'num_tokens': 179755.0, 'mean_token_accuracy': 0.6887824535369873, 'epoch': 1.19}
{'loss': 1.6101, 'grad_norm': 2.5904288291931152, 'learning_rate': 8.913477223198951e-05, 'entropy': 1.4837937653064728, 'num_tokens': 180316.0, 'mean_token_accuracy': 0.6304269582033157, 'epoch': 1.2}
{'loss': 1.1792, 'grad_norm': 2.135904550552368, 'learning_rate': 8.901362810086464e-05, 'entropy': 1.3290880024433136, 'num_tokens': 180860.0, 'mean_token_accuracy': 0.6924182176589966, 'epoch': 1.2}
{'loss': 1.0841, 'grad_norm': 2.2645106315612793, 'learning_rate': 8.889189564312417e-05, 'entropy': 1.2886396050453186, 'num_tokens': 181369.0, 'mean_token_accuracy': 0.7269788980484009, 'epoch': 1.21}
{'loss': 1.3432, 'grad_norm': 2.9765496253967285, 'learning_rate': 8.876957669449694e-05, 'entropy': 1.3315644264221191, 'num_tokens': 181859.0, 'mean_token_accuracy': 0.6789673715829849, 'epoch': 1.22}
{'loss': 1.3044, 'grad_norm': 2.849506378173828, 'learning_rate': 8.864667309955619e-05, 'entropy': 1.2769083082675934, 'num_tokens': 182331.0, 'mean_token_accuracy': 0.6852493435144424, 'epoch': 1.22}
{'loss': 0.9589, 'grad_norm': 2.3703930377960205, 'learning_rate': 8.852318671169162e-05, 'entropy': 1.1785804629325867, 'num_tokens': 182763.0, 'mean_token_accuracy': 0.7460848093032837, 'epoch': 1.23}
{'loss': 1.0038, 'grad_norm': 2.5578789710998535, 'learning_rate': 8.839911939308143e-05, 'entropy': 1.1740697920322418, 'num_tokens': 183129.0, 'mean_token_accuracy': 0.7187470495700836, 'epoch': 1.23}
{'loss': 1.181, 'grad_norm': 2.785048246383667, 'learning_rate': 8.827447301466432e-05, 'entropy': 1.3546298146247864, 'num_tokens': 183464.0, 'mean_token_accuracy': 0.6919262558221817, 'epoch': 1.24}
{'loss': 1.1864, 'grad_norm': 3.1806540489196777, 'learning_rate': 8.814924945611118e-05, 'entropy': 1.2684380412101746, 'num_tokens': 183772.0, 'mean_token_accuracy': 0.6664006561040878, 'epoch': 1.25}
{'loss': 1.4143, 'grad_norm': 1.6163581609725952, 'learning_rate': 8.802345060579684e-05, 'entropy': 1.159772902727127, 'num_tokens': 186216.0, 'mean_token_accuracy': 0.6385893374681473, 'epoch': 1.25}
{'loss': 1.2907, 'grad_norm': 2.0569546222686768, 'learning_rate': 8.789707836077149e-05, 'entropy': 1.0557668954133987, 'num_tokens': 187948.0, 'mean_token_accuracy': 0.680767685174942, 'epoch': 1.26}
{'loss': 1.1847, 'grad_norm': 1.7154020071029663, 'learning_rate': 8.777013462673217e-05, 'entropy': 1.0393365025520325, 'num_tokens': 189417.0, 'mean_token_accuracy': 0.6703139543533325, 'epoch': 1.26}
{'loss': 1.2987, 'grad_norm': 1.6873633861541748, 'learning_rate': 8.764262131799401e-05, 'entropy': 1.199025422334671, 'num_tokens': 190792.0, 'mean_token_accuracy': 0.6682634204626083, 'epoch': 1.27}
{'loss': 1.07, 'grad_norm': 2.023306131362915, 'learning_rate': 8.75145403574613e-05, 'entropy': 1.09101203083992, 'num_tokens': 192075.0, 'mean_token_accuracy': 0.7154649198055267, 'epoch': 1.28}
{'loss': 1.3855, 'grad_norm': 1.6998273134231567, 'learning_rate': 8.738589367659853e-05, 'entropy': 1.3569056987762451, 'num_tokens': 193281.0, 'mean_token_accuracy': 0.6579378694295883, 'epoch': 1.28}
{'loss': 1.5384, 'grad_norm': 1.6773700714111328, 'learning_rate': 8.725668321540128e-05, 'entropy': 1.4521048069000244, 'num_tokens': 194448.0, 'mean_token_accuracy': 0.615526482462883, 'epoch': 1.29}
{'loss': 1.2899, 'grad_norm': 1.5407508611679077, 'learning_rate': 8.7126910922367e-05, 'entropy': 1.2943491637706757, 'num_tokens': 195574.0, 'mean_token_accuracy': 0.6471401005983353, 'epoch': 1.29}
{'loss': 1.4694, 'grad_norm': 1.53977370262146, 'learning_rate': 8.699657875446551e-05, 'entropy': 1.536729782819748, 'num_tokens': 196662.0, 'mean_token_accuracy': 0.6160425692796707, 'epoch': 1.3}
{'loss': 1.5313, 'grad_norm': 1.64936101436615, 'learning_rate': 8.686568867710962e-05, 'entropy': 1.5386295914649963, 'num_tokens': 197726.0, 'mean_token_accuracy': 0.6304671764373779, 'epoch': 1.31}
{'loss': 1.3184, 'grad_norm': 1.6151942014694214, 'learning_rate': 8.673424266412538e-05, 'entropy': 1.5301429331302643, 'num_tokens': 198769.0, 'mean_token_accuracy': 0.6699783504009247, 'epoch': 1.31}
{'loss': 1.259, 'grad_norm': 2.1838603019714355, 'learning_rate': 8.660224269772237e-05, 'entropy': 1.3418115079402924, 'num_tokens': 199790.0, 'mean_token_accuracy': 0.6747948080301285, 'epoch': 1.32}
{'loss': 1.4031, 'grad_norm': 1.6748398542404175, 'learning_rate': 8.646969076846383e-05, 'entropy': 1.3732286095619202, 'num_tokens': 200768.0, 'mean_token_accuracy': 0.6608179062604904, 'epoch': 1.32}
{'loss': 1.1931, 'grad_norm': 2.100968360900879, 'learning_rate': 8.633658887523664e-05, 'entropy': 1.369848906993866, 'num_tokens': 201709.0, 'mean_token_accuracy': 0.6872353106737137, 'epoch': 1.33}
{'loss': 0.9098, 'grad_norm': 1.5184093713760376, 'learning_rate': 8.620293902522105e-05, 'entropy': 1.0147781893610954, 'num_tokens': 202624.0, 'mean_token_accuracy': 0.7692190557718277, 'epoch': 1.34}
{'loss': 1.441, 'grad_norm': 2.1523516178131104, 'learning_rate': 8.606874323386062e-05, 'entropy': 1.5245209038257599, 'num_tokens': 203492.0, 'mean_token_accuracy': 0.6611157655715942, 'epoch': 1.34}
{'loss': 1.0476, 'grad_norm': 3.0054304599761963, 'learning_rate': 8.593400352483168e-05, 'entropy': 1.091543287038803, 'num_tokens': 204343.0, 'mean_token_accuracy': 0.7495161294937134, 'epoch': 1.35}
{'loss': 1.2748, 'grad_norm': 1.757867693901062, 'learning_rate': 8.579872193001283e-05, 'entropy': 1.4378978610038757, 'num_tokens': 205167.0, 'mean_token_accuracy': 0.6784980744123459, 'epoch': 1.35}
{'loss': 1.1987, 'grad_norm': 1.8848556280136108, 'learning_rate': 8.56629004894544e-05, 'entropy': 1.4166304171085358, 'num_tokens': 205971.0, 'mean_token_accuracy': 0.6900797635316849, 'epoch': 1.36}
{'loss': 1.4871, 'grad_norm': 2.0723581314086914, 'learning_rate': 8.552654125134751e-05, 'entropy': 1.517738789319992, 'num_tokens': 206757.0, 'mean_token_accuracy': 0.6401054710149765, 'epoch': 1.37}
{'loss': 1.3318, 'grad_norm': 2.0444746017456055, 'learning_rate': 8.538964627199332e-05, 'entropy': 1.484053760766983, 'num_tokens': 207502.0, 'mean_token_accuracy': 0.6824126690626144, 'epoch': 1.37}
{'loss': 1.4527, 'grad_norm': 2.829409599304199, 'learning_rate': 8.525221761577204e-05, 'entropy': 1.5114379525184631, 'num_tokens': 208209.0, 'mean_token_accuracy': 0.6662718206644058, 'epoch': 1.38}
{'loss': 1.192, 'grad_norm': 1.8390392065048218, 'learning_rate': 8.511425735511167e-05, 'entropy': 1.3186835944652557, 'num_tokens': 208871.0, 'mean_token_accuracy': 0.7007779330015182, 'epoch': 1.38}
{'loss': 1.3115, 'grad_norm': 2.174262285232544, 'learning_rate': 8.497576757045683e-05, 'entropy': 1.4365952908992767, 'num_tokens': 209513.0, 'mean_token_accuracy': 0.6536458283662796, 'epoch': 1.39}
{'loss': 1.2759, 'grad_norm': 2.3467204570770264, 'learning_rate': 8.483675035023739e-05, 'entropy': 1.358951359987259, 'num_tokens': 210149.0, 'mean_token_accuracy': 0.6771252453327179, 'epoch': 1.4}
{'loss': 1.2318, 'grad_norm': 2.3107874393463135, 'learning_rate': 8.4697207790837e-05, 'entropy': 1.285747617483139, 'num_tokens': 210765.0, 'mean_token_accuracy': 0.6763460636138916, 'epoch': 1.4}
{'loss': 1.323, 'grad_norm': 2.067850112915039, 'learning_rate': 8.455714199656137e-05, 'entropy': 1.5687414109706879, 'num_tokens': 211370.0, 'mean_token_accuracy': 0.6871232837438583, 'epoch': 1.41}
{'loss': 1.4997, 'grad_norm': 2.202871561050415, 'learning_rate': 8.441655507960667e-05, 'entropy': 1.6364564299583435, 'num_tokens': 211966.0, 'mean_token_accuracy': 0.6755380630493164, 'epoch': 1.42}
{'loss': 1.1251, 'grad_norm': 2.2276365756988525, 'learning_rate': 8.427544916002755e-05, 'entropy': 1.2467165887355804, 'num_tokens': 212541.0, 'mean_token_accuracy': 0.7283084988594055, 'epoch': 1.42}
{'loss': 1.3691, 'grad_norm': 2.2817909717559814, 'learning_rate': 8.41338263657053e-05, 'entropy': 1.5066403150558472, 'num_tokens': 213099.0, 'mean_token_accuracy': 0.6859738528728485, 'epoch': 1.43}
{'loss': 1.2557, 'grad_norm': 2.9499080181121826, 'learning_rate': 8.399168883231564e-05, 'entropy': 1.3831834495067596, 'num_tokens': 213634.0, 'mean_token_accuracy': 0.6846067905426025, 'epoch': 1.43}
{'loss': 1.0855, 'grad_norm': 2.3065731525421143, 'learning_rate': 8.38490387032966e-05, 'entropy': 1.1910563707351685, 'num_tokens': 214157.0, 'mean_token_accuracy': 0.7320065647363663, 'epoch': 1.44}
{'loss': 1.0602, 'grad_norm': 2.42372989654541, 'learning_rate': 8.37058781298162e-05, 'entropy': 1.1663656830787659, 'num_tokens': 214667.0, 'mean_token_accuracy': 0.7391394972801208, 'epoch': 1.45}
{'loss': 1.2633, 'grad_norm': 2.635803461074829, 'learning_rate': 8.35622092707399e-05, 'entropy': 1.44778373837471, 'num_tokens': 215149.0, 'mean_token_accuracy': 0.6460992097854614, 'epoch': 1.45}
{'loss': 1.2431, 'grad_norm': 2.728095293045044, 'learning_rate': 8.341803429259817e-05, 'entropy': 1.4367583096027374, 'num_tokens': 215615.0, 'mean_token_accuracy': 0.6795727163553238, 'epoch': 1.46}
{'loss': 0.9856, 'grad_norm': 2.6904149055480957, 'learning_rate': 8.327335536955376e-05, 'entropy': 1.1847264915704727, 'num_tokens': 216066.0, 'mean_token_accuracy': 0.7454362064599991, 'epoch': 1.46}
{'loss': 1.1322, 'grad_norm': 2.9661905765533447, 'learning_rate': 8.31281746833689e-05, 'entropy': 1.2702895998954773, 'num_tokens': 216505.0, 'mean_token_accuracy': 0.7173041552305222, 'epoch': 1.47}
{'loss': 1.222, 'grad_norm': 2.8714492321014404, 'learning_rate': 8.29824944233725e-05, 'entropy': 1.1655261367559433, 'num_tokens': 216925.0, 'mean_token_accuracy': 0.7212505787611008, 'epoch': 1.48}
{'loss': 1.1628, 'grad_norm': 2.8655126094818115, 'learning_rate': 8.283631678642694e-05, 'entropy': 1.203385591506958, 'num_tokens': 217316.0, 'mean_token_accuracy': 0.7144404649734497, 'epoch': 1.48}
{'loss': 1.2475, 'grad_norm': 3.2695016860961914, 'learning_rate': 8.268964397689516e-05, 'entropy': 1.2140533924102783, 'num_tokens': 217663.0, 'mean_token_accuracy': 0.700549453496933, 'epoch': 1.49}
{'loss': 1.0666, 'grad_norm': 3.4269585609436035, 'learning_rate': 8.254247820660727e-05, 'entropy': 1.1175706386566162, 'num_tokens': 217956.0, 'mean_token_accuracy': 0.7049662917852402, 'epoch': 1.49}
{'loss': 1.2607, 'grad_norm': 1.4303935766220093, 'learning_rate': 8.239482169482726e-05, 'entropy': 1.0808742940425873, 'num_tokens': 220846.0, 'mean_token_accuracy': 0.6639408767223358, 'epoch': 1.5}
{'loss': 1.2149, 'grad_norm': 1.6184444427490234, 'learning_rate': 8.22466766682195e-05, 'entropy': 1.0453573763370514, 'num_tokens': 222789.0, 'mean_token_accuracy': 0.6653287708759308, 'epoch': 1.51}
{'loss': 1.3607, 'grad_norm': 1.5545600652694702, 'learning_rate': 8.209804536081516e-05, 'entropy': 1.2411546111106873, 'num_tokens': 224430.0, 'mean_token_accuracy': 0.6443425416946411, 'epoch': 1.51}
{'loss': 0.8112, 'grad_norm': 1.5701476335525513, 'learning_rate': 8.194893001397858e-05, 'entropy': 0.9327071905136108, 'num_tokens': 225833.0, 'mean_token_accuracy': 0.7826732993125916, 'epoch': 1.52}
{'loss': 1.1365, 'grad_norm': 1.9339350461959839, 'learning_rate': 8.179933287637342e-05, 'entropy': 1.2154854238033295, 'num_tokens': 227109.0, 'mean_token_accuracy': 0.7054640501737595, 'epoch': 1.52}
{'loss': 1.5063, 'grad_norm': 1.7727223634719849, 'learning_rate': 8.164925620392872e-05, 'entropy': 1.438150316476822, 'num_tokens': 228338.0, 'mean_token_accuracy': 0.6199021190404892, 'epoch': 1.53}
{'loss': 1.1319, 'grad_norm': 1.559585452079773, 'learning_rate': 8.149870225980498e-05, 'entropy': 1.1665003597736359, 'num_tokens': 229518.0, 'mean_token_accuracy': 0.6955012679100037, 'epoch': 1.54}
{'loss': 1.2577, 'grad_norm': 1.768940806388855, 'learning_rate': 8.134767331435991e-05, 'entropy': 1.2918634712696075, 'num_tokens': 230665.0, 'mean_token_accuracy': 0.6657298505306244, 'epoch': 1.54}
{'loss': 1.382, 'grad_norm': 1.7730056047439575, 'learning_rate': 8.11961716451143e-05, 'entropy': 1.4439948201179504, 'num_tokens': 231796.0, 'mean_token_accuracy': 0.6256440430879593, 'epoch': 1.55}
{'loss': 1.5122, 'grad_norm': 1.9531348943710327, 'learning_rate': 8.104419953671759e-05, 'entropy': 1.4845267236232758, 'num_tokens': 232869.0, 'mean_token_accuracy': 0.6211995482444763, 'epoch': 1.55}
{'loss': 1.633, 'grad_norm': 1.9163942337036133, 'learning_rate': 8.089175928091349e-05, 'entropy': 1.4983018338680267, 'num_tokens': 233887.0, 'mean_token_accuracy': 0.5955611020326614, 'epoch': 1.56}
{'loss': 1.5406, 'grad_norm': 2.0455102920532227, 'learning_rate': 8.073885317650534e-05, 'entropy': 1.5113847851753235, 'num_tokens': 234867.0, 'mean_token_accuracy': 0.6440217643976212, 'epoch': 1.57}
{'loss': 1.3783, 'grad_norm': 2.685744524002075, 'learning_rate': 8.058548352932158e-05, 'entropy': 1.4636358618736267, 'num_tokens': 235818.0, 'mean_token_accuracy': 0.6633063107728958, 'epoch': 1.57}
{'loss': 1.3342, 'grad_norm': 2.4051599502563477, 'learning_rate': 8.043165265218078e-05, 'entropy': 1.3551365733146667, 'num_tokens': 236728.0, 'mean_token_accuracy': 0.6629350036382675, 'epoch': 1.58}
{'loss': 1.4572, 'grad_norm': 1.894595980644226, 'learning_rate': 8.027736286485694e-05, 'entropy': 1.574996441602707, 'num_tokens': 237598.0, 'mean_token_accuracy': 0.6441671997308731, 'epoch': 1.58}
{'loss': 1.275, 'grad_norm': 2.1232504844665527, 'learning_rate': 8.01226164940444e-05, 'entropy': 1.2950424328446388, 'num_tokens': 238440.0, 'mean_token_accuracy': 0.6800447404384613, 'epoch': 1.59}
{'loss': 1.09, 'grad_norm': 1.789435625076294, 'learning_rate': 7.996741587332284e-05, 'entropy': 1.3112962543964386, 'num_tokens': 239242.0, 'mean_token_accuracy': 0.7028325796127319, 'epoch': 1.6}
{'loss': 1.1715, 'grad_norm': 2.0242655277252197, 'learning_rate': 7.981176334312199e-05, 'entropy': 1.257117360830307, 'num_tokens': 240026.0, 'mean_token_accuracy': 0.7231121957302094, 'epoch': 1.6}
{'loss': 1.5514, 'grad_norm': 2.1239311695098877, 'learning_rate': 7.965566125068642e-05, 'entropy': 1.5579672157764435, 'num_tokens': 240799.0, 'mean_token_accuracy': 0.6139988303184509, 'epoch': 1.61}
{'loss': 1.2578, 'grad_norm': 1.802069902420044, 'learning_rate': 7.949911195004008e-05, 'entropy': 1.4602919220924377, 'num_tokens': 241542.0, 'mean_token_accuracy': 0.6888280957937241, 'epoch': 1.61}
{'loss': 1.4041, 'grad_norm': 2.0594611167907715, 'learning_rate': 7.934211780195089e-05, 'entropy': 1.4458201825618744, 'num_tokens': 242256.0, 'mean_token_accuracy': 0.656478613615036, 'epoch': 1.62}
{'loss': 1.2872, 'grad_norm': 2.221282720565796, 'learning_rate': 7.9184681173895e-05, 'entropy': 1.4120673537254333, 'num_tokens': 242954.0, 'mean_token_accuracy': 0.6944638192653656, 'epoch': 1.63}
{'loss': 1.611, 'grad_norm': 2.2546684741973877, 'learning_rate': 7.902680444002127e-05, 'entropy': 1.5867272913455963, 'num_tokens': 243640.0, 'mean_token_accuracy': 0.6202768385410309, 'epoch': 1.63}
{'loss': 1.2656, 'grad_norm': 2.2089900970458984, 'learning_rate': 7.886848998111526e-05, 'entropy': 1.3333966732025146, 'num_tokens': 244316.0, 'mean_token_accuracy': 0.6757739931344986, 'epoch': 1.64}
{'loss': 1.3001, 'grad_norm': 2.0091867446899414, 'learning_rate': 7.870974018456352e-05, 'entropy': 1.4142038226127625, 'num_tokens': 244977.0, 'mean_token_accuracy': 0.6727272719144821, 'epoch': 1.64}
{'loss': 1.1366, 'grad_norm': 1.9087176322937012, 'learning_rate': 7.855055744431747e-05, 'entropy': 1.3103576004505157, 'num_tokens': 245628.0, 'mean_token_accuracy': 0.6955795586109161, 'epoch': 1.65}
{'loss': 1.1898, 'grad_norm': 2.3007969856262207, 'learning_rate': 7.839094416085733e-05, 'entropy': 1.334787368774414, 'num_tokens': 246254.0, 'mean_token_accuracy': 0.6942805498838425, 'epoch': 1.66}
{'loss': 1.2105, 'grad_norm': 2.0846502780914307, 'learning_rate': 7.823090274115592e-05, 'entropy': 1.2695575058460236, 'num_tokens': 246867.0, 'mean_token_accuracy': 0.7045977115631104, 'epoch': 1.66}
{'loss': 1.2275, 'grad_norm': 2.1054630279541016, 'learning_rate': 7.807043559864242e-05, 'entropy': 1.2961940169334412, 'num_tokens': 247469.0, 'mean_token_accuracy': 0.6856306344270706, 'epoch': 1.67}
{'loss': 1.4722, 'grad_norm': 2.2655701637268066, 'learning_rate': 7.790954515316584e-05, 'entropy': 1.3877536356449127, 'num_tokens': 248058.0, 'mean_token_accuracy': 0.6598680466413498, 'epoch': 1.67}
{'loss': 1.3464, 'grad_norm': 2.5855321884155273, 'learning_rate': 7.774823383095867e-05, 'entropy': 1.4928893446922302, 'num_tokens': 248619.0, 'mean_token_accuracy': 0.6408690363168716, 'epoch': 1.68}
{'loss': 1.2676, 'grad_norm': 2.4260101318359375, 'learning_rate': 7.758650406460018e-05, 'entropy': 1.374647080898285, 'num_tokens': 249171.0, 'mean_token_accuracy': 0.6934409588575363, 'epoch': 1.69}
{'loss': 1.1831, 'grad_norm': 2.2556040287017822, 'learning_rate': 7.742435829297988e-05, 'entropy': 1.2529206275939941, 'num_tokens': 249710.0, 'mean_token_accuracy': 0.7066744416952133, 'epoch': 1.69}
{'loss': 1.0326, 'grad_norm': 2.2115838527679443, 'learning_rate': 7.726179896126055e-05, 'entropy': 1.2612384557724, 'num_tokens': 250229.0, 'mean_token_accuracy': 0.7416923195123672, 'epoch': 1.7}
{'loss': 1.2616, 'grad_norm': 2.641359567642212, 'learning_rate': 7.709882852084152e-05, 'entropy': 1.3928896486759186, 'num_tokens': 250720.0, 'mean_token_accuracy': 0.7250627428293228, 'epoch': 1.7}
{'loss': 1.1716, 'grad_norm': 2.736052989959717, 'learning_rate': 7.693544942932162e-05, 'entropy': 1.3426445424556732, 'num_tokens': 251190.0, 'mean_token_accuracy': 0.7146070152521133, 'epoch': 1.71}
{'loss': 1.1015, 'grad_norm': 2.6432056427001953, 'learning_rate': 7.677166415046225e-05, 'entropy': 1.269813060760498, 'num_tokens': 251627.0, 'mean_token_accuracy': 0.725141778588295, 'epoch': 1.72}
{'loss': 0.9927, 'grad_norm': 2.8936192989349365, 'learning_rate': 7.660747515414996e-05, 'entropy': 1.1364297717809677, 'num_tokens': 252051.0, 'mean_token_accuracy': 0.7404382526874542, 'epoch': 1.72}
{'loss': 0.9752, 'grad_norm': 2.9195799827575684, 'learning_rate': 7.644288491635952e-05, 'entropy': 1.0805188417434692, 'num_tokens': 252447.0, 'mean_token_accuracy': 0.7218728959560394, 'epoch': 1.73}
{'loss': 1.2259, 'grad_norm': 3.7216832637786865, 'learning_rate': 7.627789591911634e-05, 'entropy': 1.2460293173789978, 'num_tokens': 252781.0, 'mean_token_accuracy': 0.7087912112474442, 'epoch': 1.73}
{'loss': 1.0022, 'grad_norm': 3.248323917388916, 'learning_rate': 7.611251065045918e-05, 'entropy': 1.2600956857204437, 'num_tokens': 253075.0, 'mean_token_accuracy': 0.7140324115753174, 'epoch': 1.74}
{'loss': 1.3754, 'grad_norm': 1.5412851572036743, 'learning_rate': 7.594673160440258e-05, 'entropy': 1.2077046632766724, 'num_tokens': 255258.0, 'mean_token_accuracy': 0.6511461138725281, 'epoch': 1.75}
{'loss': 1.4807, 'grad_norm': 1.5079288482666016, 'learning_rate': 7.578056128089921e-05, 'entropy': 1.3109474778175354, 'num_tokens': 257227.0, 'mean_token_accuracy': 0.6311039328575134, 'epoch': 1.75}
{'loss': 1.402, 'grad_norm': 1.5648293495178223, 'learning_rate': 7.56140021858023e-05, 'entropy': 1.3031586408615112, 'num_tokens': 259096.0, 'mean_token_accuracy': 0.645521953701973, 'epoch': 1.76}
{'loss': 1.4561, 'grad_norm': 1.963624358177185, 'learning_rate': 7.544705683082768e-05, 'entropy': 1.350146859884262, 'num_tokens': 260826.0, 'mean_token_accuracy': 0.6358702331781387, 'epoch': 1.76}
{'loss': 1.1588, 'grad_norm': 1.8645707368850708, 'learning_rate': 7.527972773351604e-05, 'entropy': 1.1636942625045776, 'num_tokens': 262410.0, 'mean_token_accuracy': 0.6956008672714233, 'epoch': 1.77}
{'loss': 1.2504, 'grad_norm': 1.5990151166915894, 'learning_rate': 7.511201741719489e-05, 'entropy': 1.275083303451538, 'num_tokens': 263823.0, 'mean_token_accuracy': 0.6661749184131622, 'epoch': 1.78}
{'loss': 1.3068, 'grad_norm': 1.7246925830841064, 'learning_rate': 7.494392841094058e-05, 'entropy': 1.2373639047145844, 'num_tokens': 265150.0, 'mean_token_accuracy': 0.6440561413764954, 'epoch': 1.78}
{'loss': 1.1605, 'grad_norm': 1.509974479675293, 'learning_rate': 7.477546324954002e-05, 'entropy': 1.268522709608078, 'num_tokens': 266461.0, 'mean_token_accuracy': 0.6939902752637863, 'epoch': 1.79}
{'loss': 1.121, 'grad_norm': 1.437000036239624, 'learning_rate': 7.460662447345265e-05, 'entropy': 1.2777235209941864, 'num_tokens': 267757.0, 'mean_token_accuracy': 0.7187987565994263, 'epoch': 1.79}
{'loss': 1.3175, 'grad_norm': 1.703313946723938, 'learning_rate': 7.443741462877202e-05, 'entropy': 1.326607197523117, 'num_tokens': 268989.0, 'mean_token_accuracy': 0.6572168618440628, 'epoch': 1.8}
{'loss': 1.4543, 'grad_norm': 1.814881682395935, 'learning_rate': 7.426783626718732e-05, 'entropy': 1.3344365060329437, 'num_tokens': 270198.0, 'mean_token_accuracy': 0.6465748697519302, 'epoch': 1.81}
{'loss': 1.4841, 'grad_norm': 1.7918376922607422, 'learning_rate': 7.409789194594507e-05, 'entropy': 1.4604476392269135, 'num_tokens': 271372.0, 'mean_token_accuracy': 0.6205241084098816, 'epoch': 1.81}
{'loss': 1.4757, 'grad_norm': 1.8075618743896484, 'learning_rate': 7.392758422781044e-05, 'entropy': 1.4072017669677734, 'num_tokens': 272511.0, 'mean_token_accuracy': 0.635262742638588, 'epoch': 1.82}
{'loss': 1.5172, 'grad_norm': 1.8178682327270508, 'learning_rate': 7.375691568102863e-05, 'entropy': 1.5423056781291962, 'num_tokens': 273601.0, 'mean_token_accuracy': 0.6274425536394119, 'epoch': 1.82}
{'loss': 0.9771, 'grad_norm': 1.754399061203003, 'learning_rate': 7.358588887928614e-05, 'entropy': 1.1519910544157028, 'num_tokens': 274642.0, 'mean_token_accuracy': 0.7542056441307068, 'epoch': 1.83}
{'loss': 1.271, 'grad_norm': 1.7184312343597412, 'learning_rate': 7.341450640167202e-05, 'entropy': 1.4190824925899506, 'num_tokens': 275643.0, 'mean_token_accuracy': 0.6786007434129715, 'epoch': 1.84}
{'loss': 1.3823, 'grad_norm': 2.2056190967559814, 'learning_rate': 7.324277083263885e-05, 'entropy': 1.3987876176834106, 'num_tokens': 276605.0, 'mean_token_accuracy': 0.6561894416809082, 'epoch': 1.84}
{'loss': 1.3457, 'grad_norm': 1.9272030591964722, 'learning_rate': 7.307068476196388e-05, 'entropy': 1.3375559449195862, 'num_tokens': 277535.0, 'mean_token_accuracy': 0.6585811227560043, 'epoch': 1.85}
{'loss': 1.1777, 'grad_norm': 1.7752646207809448, 'learning_rate': 7.289825078470993e-05, 'entropy': 1.3001608848571777, 'num_tokens': 278431.0, 'mean_token_accuracy': 0.7042372077703476, 'epoch': 1.85}
{'loss': 0.9682, 'grad_norm': 2.1713192462921143, 'learning_rate': 7.272547150118624e-05, 'entropy': 1.1472834050655365, 'num_tokens': 279303.0, 'mean_token_accuracy': 0.7477774322032928, 'epoch': 1.86}
{'loss': 1.175, 'grad_norm': 2.044156551361084, 'learning_rate': 7.255234951690932e-05, 'entropy': 1.3403066992759705, 'num_tokens': 280159.0, 'mean_token_accuracy': 0.6877752840518951, 'epoch': 1.87}
{'loss': 1.395, 'grad_norm': 2.0100479125976562, 'learning_rate': 7.237888744256357e-05, 'entropy': 1.4522035121917725, 'num_tokens': 280996.0, 'mean_token_accuracy': 0.6674066036939621, 'epoch': 1.87}
{'loss': 0.9932, 'grad_norm': 2.396144151687622, 'learning_rate': 7.220508789396197e-05, 'entropy': 1.1134671792387962, 'num_tokens': 281811.0, 'mean_token_accuracy': 0.7646675407886505, 'epoch': 1.88}
{'loss': 1.3841, 'grad_norm': 1.9179590940475464, 'learning_rate': 7.203095349200666e-05, 'entropy': 1.4307136833667755, 'num_tokens': 282614.0, 'mean_token_accuracy': 0.6591519862413406, 'epoch': 1.88}
{'loss': 1.3869, 'grad_norm': 2.552586793899536, 'learning_rate': 7.185648686264934e-05, 'entropy': 1.4579284489154816, 'num_tokens': 283391.0, 'mean_token_accuracy': 0.6363347321748734, 'epoch': 1.89}
{'loss': 1.5251, 'grad_norm': 2.4693737030029297, 'learning_rate': 7.168169063685171e-05, 'entropy': 1.5808922052383423, 'num_tokens': 284142.0, 'mean_token_accuracy': 0.6334308236837387, 'epoch': 1.9}
{'loss': 1.3955, 'grad_norm': 2.010483741760254, 'learning_rate': 7.15065674505458e-05, 'entropy': 1.4782294929027557, 'num_tokens': 284863.0, 'mean_token_accuracy': 0.6620620787143707, 'epoch': 1.9}
{'loss': 1.2882, 'grad_norm': 2.55926513671875, 'learning_rate': 7.13311199445942e-05, 'entropy': 1.335726112127304, 'num_tokens': 285551.0, 'mean_token_accuracy': 0.6708922237157822, 'epoch': 1.91}
{'loss': 1.1828, 'grad_norm': 2.0772817134857178, 'learning_rate': 7.115535076475031e-05, 'entropy': 1.3305081874132156, 'num_tokens': 286222.0, 'mean_token_accuracy': 0.7045878171920776, 'epoch': 1.91}
{'loss': 1.1973, 'grad_norm': 3.6394574642181396, 'learning_rate': 7.09792625616183e-05, 'entropy': 1.2491894513368607, 'num_tokens': 286877.0, 'mean_token_accuracy': 0.6924995630979538, 'epoch': 1.92}
{'loss': 1.1493, 'grad_norm': 2.014920473098755, 'learning_rate': 7.080285799061324e-05, 'entropy': 1.2875721752643585, 'num_tokens': 287498.0, 'mean_token_accuracy': 0.7016797810792923, 'epoch': 1.93}
{'loss': 1.0306, 'grad_norm': 2.330594778060913, 'learning_rate': 7.06261397119211e-05, 'entropy': 1.1911025941371918, 'num_tokens': 288088.0, 'mean_token_accuracy': 0.728926032781601, 'epoch': 1.93}
{'loss': 1.2429, 'grad_norm': 2.3738694190979004, 'learning_rate': 7.044911039045847e-05, 'entropy': 1.428299456834793, 'num_tokens': 288659.0, 'mean_token_accuracy': 0.6837844401597977, 'epoch': 1.94}
{'loss': 1.1461, 'grad_norm': 2.178236722946167, 'learning_rate': 7.027177269583256e-05, 'entropy': 1.2943442165851593, 'num_tokens': 289208.0, 'mean_token_accuracy': 0.7033692449331284, 'epoch': 1.94}
{'loss': 1.337, 'grad_norm': 2.6979727745056152, 'learning_rate': 7.009412930230084e-05, 'entropy': 1.3791843056678772, 'num_tokens': 289728.0, 'mean_token_accuracy': 0.6860758662223816, 'epoch': 1.95}
{'loss': 1.2926, 'grad_norm': 2.6353440284729004, 'learning_rate': 6.991618288873066e-05, 'entropy': 1.3696454465389252, 'num_tokens': 290218.0, 'mean_token_accuracy': 0.6803992390632629, 'epoch': 1.96}
{'loss': 1.2447, 'grad_norm': 2.8331832885742188, 'learning_rate': 6.973793613855902e-05, 'entropy': 1.4395420551300049, 'num_tokens': 290682.0, 'mean_token_accuracy': 0.7044126093387604, 'epoch': 1.96}
{'loss': 1.0363, 'grad_norm': 2.677757978439331, 'learning_rate': 6.955939173975191e-05, 'entropy': 1.1938076615333557, 'num_tokens': 291117.0, 'mean_token_accuracy': 0.7305754125118256, 'epoch': 1.97}
{'loss': 1.212, 'grad_norm': 3.0835044384002686, 'learning_rate': 6.938055238476395e-05, 'entropy': 1.2336171865463257, 'num_tokens': 291535.0, 'mean_token_accuracy': 0.717558354139328, 'epoch': 1.97}
{'loss': 1.1577, 'grad_norm': 3.0286920070648193, 'learning_rate': 6.920142077049766e-05, 'entropy': 1.2943689823150635, 'num_tokens': 291885.0, 'mean_token_accuracy': 0.6758386641740799, 'epoch': 1.98}
{'loss': 1.2984, 'grad_norm': 3.2300257682800293, 'learning_rate': 6.902199959826286e-05, 'entropy': 1.2735935747623444, 'num_tokens': 292205.0, 'mean_token_accuracy': 0.7057927995920181, 'epoch': 1.99}
{'loss': 1.1989, 'grad_norm': 1.432762622833252, 'learning_rate': 6.88422915737359e-05, 'entropy': 1.1576189696788788, 'num_tokens': 293692.0, 'mean_token_accuracy': 0.6845733821392059, 'epoch': 1.99}
{'loss': 1.5477, 'grad_norm': 3.102828025817871, 'learning_rate': 6.866229940691888e-05, 'entropy': 1.3589507639408112, 'num_tokens': 294421.0, 'mean_token_accuracy': 0.6128910481929779, 'epoch': 2.0}
{'loss': 1.1798, 'grad_norm': 6.910417079925537, 'learning_rate': 6.848202581209875e-05, 'entropy': 1.310160756111145, 'num_tokens': 294470.0, 'mean_token_accuracy': 0.625, 'epoch': 2.0}
{'loss': 1.0583, 'grad_norm': 1.027564287185669, 'learning_rate': 6.830147350780645e-05, 'entropy': 1.054894581437111, 'num_tokens': 297419.0, 'mean_token_accuracy': 0.7116810977458954, 'epoch': 2.01}
{'loss': 1.154, 'grad_norm': 1.2407987117767334, 'learning_rate': 6.812064521677579e-05, 'entropy': 1.2606667578220367, 'num_tokens': 299521.0, 'mean_token_accuracy': 0.6801898330450058, 'epoch': 2.01}
{'loss': 1.0444, 'grad_norm': 1.3765536546707153, 'learning_rate': 6.793954366590257e-05, 'entropy': 1.0924577414989471, 'num_tokens': 301151.0, 'mean_token_accuracy': 0.7080784887075424, 'epoch': 2.02}
{'loss': 0.8468, 'grad_norm': 1.5371640920639038, 'learning_rate': 6.775817158620328e-05, 'entropy': 1.0777437388896942, 'num_tokens': 302563.0, 'mean_token_accuracy': 0.7630569636821747, 'epoch': 2.02}
{'loss': 0.9434, 'grad_norm': 1.5827298164367676, 'learning_rate': 6.7576531712774e-05, 'entropy': 1.0793745666742325, 'num_tokens': 303876.0, 'mean_token_accuracy': 0.7552025467157364, 'epoch': 2.03}
{'loss': 1.2464, 'grad_norm': 1.942623496055603, 'learning_rate': 6.739462678474917e-05, 'entropy': 1.2761900126934052, 'num_tokens': 305118.0, 'mean_token_accuracy': 0.6950299590826035, 'epoch': 2.04}
{'loss': 1.0271, 'grad_norm': 1.828399896621704, 'learning_rate': 6.721245954526025e-05, 'entropy': 1.2235166132450104, 'num_tokens': 306311.0, 'mean_token_accuracy': 0.7212849855422974, 'epoch': 2.04}
{'loss': 0.7768, 'grad_norm': 1.977196455001831, 'learning_rate': 6.703003274139438e-05, 'entropy': 0.9907469302415848, 'num_tokens': 307475.0, 'mean_token_accuracy': 0.7774958461523056, 'epoch': 2.05}
{'loss': 0.9181, 'grad_norm': 1.888717770576477, 'learning_rate': 6.684734912415289e-05, 'entropy': 0.984191507101059, 'num_tokens': 308603.0, 'mean_token_accuracy': 0.7521322965621948, 'epoch': 2.05}
{'loss': 0.8214, 'grad_norm': 1.9612727165222168, 'learning_rate': 6.666441144840994e-05, 'entropy': 0.9859774187207222, 'num_tokens': 309671.0, 'mean_token_accuracy': 0.7840134054422379, 'epoch': 2.06}
{'loss': 0.9874, 'grad_norm': 2.392993688583374, 'learning_rate': 6.648122247287083e-05, 'entropy': 1.1009299159049988, 'num_tokens': 310716.0, 'mean_token_accuracy': 0.7137384414672852, 'epoch': 2.07}
{'loss': 1.2691, 'grad_norm': 2.486894130706787, 'learning_rate': 6.62977849600305e-05, 'entropy': 1.251889556646347, 'num_tokens': 311731.0, 'mean_token_accuracy': 0.6686168015003204, 'epoch': 2.07}
{'loss': 0.7903, 'grad_norm': 2.365802764892578, 'learning_rate': 6.611410167613184e-05, 'entropy': 0.8699994906783104, 'num_tokens': 312693.0, 'mean_token_accuracy': 0.7896087616682053, 'epoch': 2.08}
{'loss': 0.9607, 'grad_norm': 2.4359304904937744, 'learning_rate': 6.593017539112396e-05, 'entropy': 0.9600937813520432, 'num_tokens': 313612.0, 'mean_token_accuracy': 0.7499500066041946, 'epoch': 2.08}
{'loss': 0.7672, 'grad_norm': 2.4484152793884277, 'learning_rate': 6.574600887862043e-05, 'entropy': 0.8201890587806702, 'num_tokens': 314500.0, 'mean_token_accuracy': 0.8001440465450287, 'epoch': 2.09}
{'loss': 0.6484, 'grad_norm': 2.1832058429718018, 'learning_rate': 6.55616049158575e-05, 'entropy': 0.8816795684397221, 'num_tokens': 315371.0, 'mean_token_accuracy': 0.8350456953048706, 'epoch': 2.1}
{'loss': 0.5185, 'grad_norm': 2.3823888301849365, 'learning_rate': 6.537696628365212e-05, 'entropy': 0.6849931590259075, 'num_tokens': 316224.0, 'mean_token_accuracy': 0.8622552901506424, 'epoch': 2.1}
{'loss': 0.8329, 'grad_norm': 2.691432476043701, 'learning_rate': 6.51920957663601e-05, 'entropy': 0.9636730812489986, 'num_tokens': 317057.0, 'mean_token_accuracy': 0.7989512830972672, 'epoch': 2.11}
{'loss': 0.7485, 'grad_norm': 2.6393258571624756, 'learning_rate': 6.500699615183408e-05, 'entropy': 0.8529583811759949, 'num_tokens': 317865.0, 'mean_token_accuracy': 0.7972636818885803, 'epoch': 2.11}
{'loss': 0.907, 'grad_norm': 2.6522655487060547, 'learning_rate': 6.482167023138147e-05, 'entropy': 1.0363140255212784, 'num_tokens': 318659.0, 'mean_token_accuracy': 0.7531537264585495, 'epoch': 2.12}
{'loss': 0.7588, 'grad_norm': 2.415259599685669, 'learning_rate': 6.46361207997224e-05, 'entropy': 0.9201224893331528, 'num_tokens': 319412.0, 'mean_token_accuracy': 0.8026551157236099, 'epoch': 2.13}
{'loss': 0.971, 'grad_norm': 2.542139768600464, 'learning_rate': 6.445035065494754e-05, 'entropy': 1.099446028470993, 'num_tokens': 320133.0, 'mean_token_accuracy': 0.7251468598842621, 'epoch': 2.13}
{'loss': 1.0411, 'grad_norm': 2.888284683227539, 'learning_rate': 6.426436259847595e-05, 'entropy': 1.1074170619249344, 'num_tokens': 320832.0, 'mean_token_accuracy': 0.739930585026741, 'epoch': 2.14}
{'loss': 0.6586, 'grad_norm': 2.4250645637512207, 'learning_rate': 6.407815943501274e-05, 'entropy': 0.7997961044311523, 'num_tokens': 321514.0, 'mean_token_accuracy': 0.8022972643375397, 'epoch': 2.14}
{'loss': 0.7661, 'grad_norm': 2.5154762268066406, 'learning_rate': 6.389174397250693e-05, 'entropy': 0.9638670086860657, 'num_tokens': 322182.0, 'mean_token_accuracy': 0.7819211930036545, 'epoch': 2.15}
{'loss': 0.5631, 'grad_norm': 2.2333829402923584, 'learning_rate': 6.370511902210897e-05, 'entropy': 0.7425584346055984, 'num_tokens': 322840.0, 'mean_token_accuracy': 0.8607567846775055, 'epoch': 2.16}
{'loss': 0.6928, 'grad_norm': 2.7148947715759277, 'learning_rate': 6.351828739812836e-05, 'entropy': 0.8997579365968704, 'num_tokens': 323486.0, 'mean_token_accuracy': 0.7835783958435059, 'epoch': 2.16}
{'loss': 0.6894, 'grad_norm': 2.3223884105682373, 'learning_rate': 6.33312519179913e-05, 'entropy': 0.9204305112361908, 'num_tokens': 324114.0, 'mean_token_accuracy': 0.8017436265945435, 'epoch': 2.17}
{'loss': 0.7477, 'grad_norm': 2.4538862705230713, 'learning_rate': 6.31440154021981e-05, 'entropy': 0.9156483113765717, 'num_tokens': 324718.0, 'mean_token_accuracy': 0.788423091173172, 'epoch': 2.17}
{'loss': 0.6226, 'grad_norm': 2.494455099105835, 'learning_rate': 6.295658067428077e-05, 'entropy': 0.74310402572155, 'num_tokens': 325301.0, 'mean_token_accuracy': 0.8499939292669296, 'epoch': 2.18}
{'loss': 0.7853, 'grad_norm': 2.5814948081970215, 'learning_rate': 6.276895056076022e-05, 'entropy': 0.9028007984161377, 'num_tokens': 325875.0, 'mean_token_accuracy': 0.7945580780506134, 'epoch': 2.19}
{'loss': 0.8117, 'grad_norm': 2.8562204837799072, 'learning_rate': 6.258112789110395e-05, 'entropy': 0.9704578816890717, 'num_tokens': 326437.0, 'mean_token_accuracy': 0.7849563211202621, 'epoch': 2.19}
{'loss': 0.5413, 'grad_norm': 2.92559814453125, 'learning_rate': 6.239311549768311e-05, 'entropy': 0.7571094334125519, 'num_tokens': 326984.0, 'mean_token_accuracy': 0.8365258574485779, 'epoch': 2.2}
{'loss': 0.45, 'grad_norm': 2.220228433609009, 'learning_rate': 6.220491621572989e-05, 'entropy': 0.6358883306384087, 'num_tokens': 327511.0, 'mean_token_accuracy': 0.9029641300439835, 'epoch': 2.2}
{'loss': 0.67, 'grad_norm': 2.988724708557129, 'learning_rate': 6.20165328832948e-05, 'entropy': 0.8299094587564468, 'num_tokens': 328021.0, 'mean_token_accuracy': 0.8229598551988602, 'epoch': 2.21}
{'loss': 0.7244, 'grad_norm': 3.3080999851226807, 'learning_rate': 6.182796834120386e-05, 'entropy': 0.8587799966335297, 'num_tokens': 328512.0, 'mean_token_accuracy': 0.8148215264081955, 'epoch': 2.22}
{'loss': 0.43, 'grad_norm': 3.1426804065704346, 'learning_rate': 6.163922543301565e-05, 'entropy': 0.6172008663415909, 'num_tokens': 328982.0, 'mean_token_accuracy': 0.8738539814949036, 'epoch': 2.22}
{'loss': 0.4331, 'grad_norm': 2.8736484050750732, 'learning_rate': 6.145030700497857e-05, 'entropy': 0.6759512722492218, 'num_tokens': 329413.0, 'mean_token_accuracy': 0.8597171753644943, 'epoch': 2.23}
{'loss': 0.4856, 'grad_norm': 3.2930147647857666, 'learning_rate': 6.126121590598788e-05, 'entropy': 0.7311371266841888, 'num_tokens': 329815.0, 'mean_token_accuracy': 0.8667093515396118, 'epoch': 2.23}
{'loss': 0.4568, 'grad_norm': 3.439429759979248, 'learning_rate': 6.10719549875427e-05, 'entropy': 0.6360659599304199, 'num_tokens': 330148.0, 'mean_token_accuracy': 0.8813523352146149, 'epoch': 2.24}
{'loss': 0.3695, 'grad_norm': 3.6468756198883057, 'learning_rate': 6.088252710370302e-05, 'entropy': 0.513482041656971, 'num_tokens': 330438.0, 'mean_token_accuracy': 0.8783462643623352, 'epoch': 2.25}
{'loss': 1.078, 'grad_norm': 2.063812017440796, 'learning_rate': 6.069293511104672e-05, 'entropy': 0.8654586225748062, 'num_tokens': 332977.0, 'mean_token_accuracy': 0.7052180618047714, 'epoch': 2.25}
{'loss': 0.8717, 'grad_norm': 2.264697551727295, 'learning_rate': 6.0503181868626394e-05, 'entropy': 0.7532070726156235, 'num_tokens': 335090.0, 'mean_token_accuracy': 0.7571559399366379, 'epoch': 2.26}
{'loss': 1.0481, 'grad_norm': 1.9525940418243408, 'learning_rate': 6.031327023792629e-05, 'entropy': 0.8930246382951736, 'num_tokens': 337143.0, 'mean_token_accuracy': 0.7136989384889603, 'epoch': 2.26}
{'loss': 1.1844, 'grad_norm': 2.0394458770751953, 'learning_rate': 6.012320308281919e-05, 'entropy': 1.0778292268514633, 'num_tokens': 339004.0, 'mean_token_accuracy': 0.6828449219465256, 'epoch': 2.27}
{'loss': 0.8351, 'grad_norm': 1.8611336946487427, 'learning_rate': 5.993298326952318e-05, 'entropy': 0.8271652162075043, 'num_tokens': 340693.0, 'mean_token_accuracy': 0.7704188227653503, 'epoch': 2.28}
{'loss': 0.7179, 'grad_norm': 1.9055932760238647, 'learning_rate': 5.974261366655841e-05, 'entropy': 0.8271828144788742, 'num_tokens': 342184.0, 'mean_token_accuracy': 0.7924222499132156, 'epoch': 2.28}
{'loss': 0.8383, 'grad_norm': 2.2976410388946533, 'learning_rate': 5.955209714470388e-05, 'entropy': 0.9274996668100357, 'num_tokens': 343507.0, 'mean_token_accuracy': 0.7644967436790466, 'epoch': 2.29}
{'loss': 1.0985, 'grad_norm': 2.453906774520874, 'learning_rate': 5.9361436576954157e-05, 'entropy': 1.181168407201767, 'num_tokens': 344721.0, 'mean_token_accuracy': 0.6979991346597672, 'epoch': 2.29}
{'loss': 1.0281, 'grad_norm': 2.3586161136627197, 'learning_rate': 5.9170634838475955e-05, 'entropy': 1.1282247751951218, 'num_tokens': 345878.0, 'mean_token_accuracy': 0.7153402715921402, 'epoch': 2.3}
{'loss': 0.9733, 'grad_norm': 2.3794450759887695, 'learning_rate': 5.897969480656491e-05, 'entropy': 1.1434669494628906, 'num_tokens': 346965.0, 'mean_token_accuracy': 0.7263156324625015, 'epoch': 2.31}
{'loss': 0.8161, 'grad_norm': 2.199150323867798, 'learning_rate': 5.8788619360602126e-05, 'entropy': 0.9323870837688446, 'num_tokens': 348014.0, 'mean_token_accuracy': 0.7985249310731888, 'epoch': 2.31}
{'loss': 0.8459, 'grad_norm': 2.6193413734436035, 'learning_rate': 5.859741138201068e-05, 'entropy': 0.8780126161873341, 'num_tokens': 349024.0, 'mean_token_accuracy': 0.7764949798583984, 'epoch': 2.32}
{'loss': 0.8337, 'grad_norm': 2.358893871307373, 'learning_rate': 5.8406073754212355e-05, 'entropy': 0.9675257652997971, 'num_tokens': 350008.0, 'mean_token_accuracy': 0.7671336382627487, 'epoch': 2.32}
{'loss': 0.9538, 'grad_norm': 2.6830127239227295, 'learning_rate': 5.8214609362583974e-05, 'entropy': 1.0267654061317444, 'num_tokens': 350961.0, 'mean_token_accuracy': 0.7421420216560364, 'epoch': 2.33}
{'loss': 0.7378, 'grad_norm': 2.3327841758728027, 'learning_rate': 5.8023021094413966e-05, 'entropy': 0.8494122065603733, 'num_tokens': 351893.0, 'mean_token_accuracy': 0.8152628242969513, 'epoch': 2.34}
{'loss': 0.5702, 'grad_norm': 1.9984134435653687, 'learning_rate': 5.783131183885885e-05, 'entropy': 0.679248945787549, 'num_tokens': 352793.0, 'mean_token_accuracy': 0.8670873790979385, 'epoch': 2.34}
{'loss': 0.8591, 'grad_norm': 2.611346483230591, 'learning_rate': 5.763948448689963e-05, 'entropy': 0.9708370715379715, 'num_tokens': 353645.0, 'mean_token_accuracy': 0.763859286904335, 'epoch': 2.35}
{'loss': 0.6879, 'grad_norm': 2.2294631004333496, 'learning_rate': 5.744754193129818e-05, 'entropy': 0.8189766928553581, 'num_tokens': 354478.0, 'mean_token_accuracy': 0.8169009685516357, 'epoch': 2.35}
{'loss': 0.7692, 'grad_norm': 2.601174831390381, 'learning_rate': 5.725548706655368e-05, 'entropy': 0.8640232682228088, 'num_tokens': 355274.0, 'mean_token_accuracy': 0.7806253731250763, 'epoch': 2.36}
{'loss': 0.9928, 'grad_norm': 2.6631157398223877, 'learning_rate': 5.706332278885893e-05, 'entropy': 1.0725084990262985, 'num_tokens': 356048.0, 'mean_token_accuracy': 0.7256034463644028, 'epoch': 2.37}
{'loss': 0.9787, 'grad_norm': 2.784010171890259, 'learning_rate': 5.687105199605667e-05, 'entropy': 1.0670721232891083, 'num_tokens': 356787.0, 'mean_token_accuracy': 0.7447159588336945, 'epoch': 2.37}
{'loss': 1.0386, 'grad_norm': 3.255580425262451, 'learning_rate': 5.6678677587595884e-05, 'entropy': 0.9787863790988922, 'num_tokens': 357482.0, 'mean_token_accuracy': 0.7494503855705261, 'epoch': 2.38}
{'loss': 0.7723, 'grad_norm': 3.1310346126556396, 'learning_rate': 5.648620246448813e-05, 'entropy': 0.8340328931808472, 'num_tokens': 358138.0, 'mean_token_accuracy': 0.7889997512102127, 'epoch': 2.38}
{'loss': 0.7347, 'grad_norm': 3.0052897930145264, 'learning_rate': 5.629362952926367e-05, 'entropy': 0.9026321023702621, 'num_tokens': 358778.0, 'mean_token_accuracy': 0.8097842484712601, 'epoch': 2.39}
{'loss': 0.7951, 'grad_norm': 2.9499659538269043, 'learning_rate': 5.610096168592785e-05, 'entropy': 0.8970423191785812, 'num_tokens': 359406.0, 'mean_token_accuracy': 0.7724642902612686, 'epoch': 2.4}
{'loss': 0.7638, 'grad_norm': 3.0246832370758057, 'learning_rate': 5.590820183991716e-05, 'entropy': 0.9137683510780334, 'num_tokens': 360023.0, 'mean_token_accuracy': 0.7684725522994995, 'epoch': 2.4}
{'loss': 0.77, 'grad_norm': 2.6990435123443604, 'learning_rate': 5.571535289805556e-05, 'entropy': 0.8995579481124878, 'num_tokens': 360630.0, 'mean_token_accuracy': 0.7894106507301331, 'epoch': 2.41}
{'loss': 0.7648, 'grad_norm': 2.959888458251953, 'learning_rate': 5.552241776851055e-05, 'entropy': 0.891539677977562, 'num_tokens': 361231.0, 'mean_token_accuracy': 0.7922259569168091, 'epoch': 2.42}
{'loss': 0.745, 'grad_norm': 3.074557065963745, 'learning_rate': 5.5329399360749336e-05, 'entropy': 0.9180279970169067, 'num_tokens': 361826.0, 'mean_token_accuracy': 0.7919367551803589, 'epoch': 2.42}
{'loss': 0.7353, 'grad_norm': 2.7407066822052, 'learning_rate': 5.513630058549497e-05, 'entropy': 0.8733026832342148, 'num_tokens': 362401.0, 'mean_token_accuracy': 0.8026565164327621, 'epoch': 2.43}
{'loss': 0.8743, 'grad_norm': 3.1847710609436035, 'learning_rate': 5.494312435468244e-05, 'entropy': 1.0491296350955963, 'num_tokens': 362964.0, 'mean_token_accuracy': 0.7548818141222, 'epoch': 2.43}
{'loss': 0.6751, 'grad_norm': 2.867133617401123, 'learning_rate': 5.474987358141478e-05, 'entropy': 0.9132025092840195, 'num_tokens': 363509.0, 'mean_token_accuracy': 0.8149212002754211, 'epoch': 2.44}
{'loss': 0.7301, 'grad_norm': 3.600632429122925, 'learning_rate': 5.4556551179919134e-05, 'entropy': 0.9006263017654419, 'num_tokens': 364039.0, 'mean_token_accuracy': 0.8019086271524429, 'epoch': 2.45}
{'loss': 0.7367, 'grad_norm': 3.465524196624756, 'learning_rate': 5.436316006550275e-05, 'entropy': 0.8550962060689926, 'num_tokens': 364551.0, 'mean_token_accuracy': 0.7951341420412064, 'epoch': 2.45}
{'loss': 0.7152, 'grad_norm': 4.069916248321533, 'learning_rate': 5.416970315450911e-05, 'entropy': 0.922579362988472, 'num_tokens': 365038.0, 'mean_token_accuracy': 0.805262103676796, 'epoch': 2.46}
{'loss': 0.7566, 'grad_norm': 3.844559669494629, 'learning_rate': 5.3976183364273946e-05, 'entropy': 0.8854574859142303, 'num_tokens': 365509.0, 'mean_token_accuracy': 0.8164269030094147, 'epoch': 2.46}
{'loss': 0.4947, 'grad_norm': 3.237761974334717, 'learning_rate': 5.3782603613081137e-05, 'entropy': 0.7460553646087646, 'num_tokens': 365961.0, 'mean_token_accuracy': 0.868796244263649, 'epoch': 2.47}
{'loss': 0.4219, 'grad_norm': 3.3295304775238037, 'learning_rate': 5.358896682011881e-05, 'entropy': 0.5966678261756897, 'num_tokens': 366375.0, 'mean_token_accuracy': 0.8931808322668076, 'epoch': 2.48}
{'loss': 0.4555, 'grad_norm': 3.560137987136841, 'learning_rate': 5.339527590543529e-05, 'entropy': 0.6975918412208557, 'num_tokens': 366752.0, 'mean_token_accuracy': 0.8799286335706711, 'epoch': 2.48}
{'loss': 0.5543, 'grad_norm': 4.4135284423828125, 'learning_rate': 5.3201533789895065e-05, 'entropy': 0.6786891967058182, 'num_tokens': 367086.0, 'mean_token_accuracy': 0.8481282442808151, 'epoch': 2.49}
{'loss': 0.4031, 'grad_norm': 3.3145153522491455, 'learning_rate': 5.300774339513468e-05, 'entropy': 0.6797176897525787, 'num_tokens': 367385.0, 'mean_token_accuracy': 0.877181351184845, 'epoch': 2.49}
{'loss': 1.0403, 'grad_norm': 2.172595977783203, 'learning_rate': 5.28139076435188e-05, 'entropy': 0.9161618947982788, 'num_tokens': 369651.0, 'mean_token_accuracy': 0.7045902162790298, 'epoch': 2.5}
{'loss': 0.7565, 'grad_norm': 2.076124429702759, 'learning_rate': 5.2620029458096045e-05, 'entropy': 0.8075636848807335, 'num_tokens': 371335.0, 'mean_token_accuracy': 0.800863116979599, 'epoch': 2.51}
{'loss': 0.9017, 'grad_norm': 2.1886343955993652, 'learning_rate': 5.242611176255492e-05, 'entropy': 0.8575737923383713, 'num_tokens': 372918.0, 'mean_token_accuracy': 0.7439811080694199, 'epoch': 2.51}
{'loss': 0.9421, 'grad_norm': 2.5007739067077637, 'learning_rate': 5.2232157481179766e-05, 'entropy': 0.8384963572025299, 'num_tokens': 374393.0, 'mean_token_accuracy': 0.7526317089796066, 'epoch': 2.52}
{'loss': 0.8308, 'grad_norm': 2.377159595489502, 'learning_rate': 5.2038169538806644e-05, 'entropy': 0.7951099127531052, 'num_tokens': 375784.0, 'mean_token_accuracy': 0.7630963027477264, 'epoch': 2.52}
{'loss': 0.721, 'grad_norm': 2.386518955230713, 'learning_rate': 5.184415086077922e-05, 'entropy': 0.8479022234678268, 'num_tokens': 377088.0, 'mean_token_accuracy': 0.7878430485725403, 'epoch': 2.53}
{'loss': 0.9052, 'grad_norm': 2.508679151535034, 'learning_rate': 5.1650104372904615e-05, 'entropy': 0.9433368146419525, 'num_tokens': 378366.0, 'mean_token_accuracy': 0.7510720491409302, 'epoch': 2.54}
{'loss': 1.014, 'grad_norm': 2.3392887115478516, 'learning_rate': 5.1456033001409395e-05, 'entropy': 1.0301789939403534, 'num_tokens': 379622.0, 'mean_token_accuracy': 0.7218741774559021, 'epoch': 2.54}
{'loss': 0.9817, 'grad_norm': 2.4737346172332764, 'learning_rate': 5.12619396728953e-05, 'entropy': 0.8873402774333954, 'num_tokens': 380812.0, 'mean_token_accuracy': 0.7444611340761185, 'epoch': 2.55}
{'loss': 0.8354, 'grad_norm': 2.3835322856903076, 'learning_rate': 5.1067827314295234e-05, 'entropy': 0.9593373537063599, 'num_tokens': 381958.0, 'mean_token_accuracy': 0.7616014927625656, 'epoch': 2.55}
{'loss': 0.8761, 'grad_norm': 2.4392311573028564, 'learning_rate': 5.087369885282908e-05, 'entropy': 1.0099204927682877, 'num_tokens': 383087.0, 'mean_token_accuracy': 0.7628013491630554, 'epoch': 2.56}
{'loss': 1.0092, 'grad_norm': 2.282017707824707, 'learning_rate': 5.067955721595947e-05, 'entropy': 1.0636009871959686, 'num_tokens': 384171.0, 'mean_token_accuracy': 0.7374050617218018, 'epoch': 2.57}
{'loss': 0.8533, 'grad_norm': 2.4756293296813965, 'learning_rate': 5.0485405331347834e-05, 'entropy': 0.9511232972145081, 'num_tokens': 385228.0, 'mean_token_accuracy': 0.7736513614654541, 'epoch': 2.57}
{'loss': 0.9031, 'grad_norm': 2.4026594161987305, 'learning_rate': 5.0291246126810065e-05, 'entropy': 1.0032212436199188, 'num_tokens': 386258.0, 'mean_token_accuracy': 0.7402098178863525, 'epoch': 2.58}
{'loss': 0.7094, 'grad_norm': 2.15195369720459, 'learning_rate': 5.0097082530272463e-05, 'entropy': 0.8690044432878494, 'num_tokens': 387260.0, 'mean_token_accuracy': 0.7974337786436081, 'epoch': 2.58}
{'loss': 0.8605, 'grad_norm': 2.518484592437744, 'learning_rate': 4.990291746972755e-05, 'entropy': 0.9855846390128136, 'num_tokens': 388214.0, 'mean_token_accuracy': 0.7560178339481354, 'epoch': 2.59}
{'loss': 0.5535, 'grad_norm': 2.007296085357666, 'learning_rate': 4.9708753873189954e-05, 'entropy': 0.6329593360424042, 'num_tokens': 389135.0, 'mean_token_accuracy': 0.8479811102151871, 'epoch': 2.6}
{'loss': 0.9275, 'grad_norm': 2.521385669708252, 'learning_rate': 4.951459466865218e-05, 'entropy': 0.9975563138723373, 'num_tokens': 390036.0, 'mean_token_accuracy': 0.7391762435436249, 'epoch': 2.6}
{'loss': 0.8248, 'grad_norm': 2.535379648208618, 'learning_rate': 4.932044278404055e-05, 'entropy': 0.9529252201318741, 'num_tokens': 390926.0, 'mean_token_accuracy': 0.7741471230983734, 'epoch': 2.61}
{'loss': 0.8084, 'grad_norm': 2.7324306964874268, 'learning_rate': 4.912630114717094e-05, 'entropy': 0.8721564039587975, 'num_tokens': 391788.0, 'mean_token_accuracy': 0.7769270092248917, 'epoch': 2.61}
{'loss': 0.8828, 'grad_norm': 2.590003490447998, 'learning_rate': 4.8932172685704764e-05, 'entropy': 1.0006208717823029, 'num_tokens': 392629.0, 'mean_token_accuracy': 0.7740018516778946, 'epoch': 2.62}
{'loss': 0.9332, 'grad_norm': 2.8991856575012207, 'learning_rate': 4.8738060327104714e-05, 'entropy': 0.9479070901870728, 'num_tokens': 393447.0, 'mean_token_accuracy': 0.7680883556604385, 'epoch': 2.63}
{'loss': 0.8295, 'grad_norm': 2.5757527351379395, 'learning_rate': 4.854396699859061e-05, 'entropy': 1.0255050957202911, 'num_tokens': 394251.0, 'mean_token_accuracy': 0.7638462036848068, 'epoch': 2.63}
{'loss': 0.8682, 'grad_norm': 2.8188490867614746, 'learning_rate': 4.8349895627095404e-05, 'entropy': 1.016544833779335, 'num_tokens': 395036.0, 'mean_token_accuracy': 0.7749172449111938, 'epoch': 2.64}
{'loss': 0.7932, 'grad_norm': 2.5286307334899902, 'learning_rate': 4.815584913922079e-05, 'entropy': 0.9788650870323181, 'num_tokens': 395792.0, 'mean_token_accuracy': 0.7856784015893936, 'epoch': 2.64}
{'loss': 0.9911, 'grad_norm': 2.9919393062591553, 'learning_rate': 4.796183046119337e-05, 'entropy': 1.1715507209300995, 'num_tokens': 396518.0, 'mean_token_accuracy': 0.749231606721878, 'epoch': 2.65}
{'loss': 0.77, 'grad_norm': 2.8424432277679443, 'learning_rate': 4.7767842518820245e-05, 'entropy': 0.8320596367120743, 'num_tokens': 397212.0, 'mean_token_accuracy': 0.8022332489490509, 'epoch': 2.66}
{'loss': 0.8488, 'grad_norm': 3.0224714279174805, 'learning_rate': 4.757388823744508e-05, 'entropy': 1.009948953986168, 'num_tokens': 397877.0, 'mean_token_accuracy': 0.7760649770498276, 'epoch': 2.66}
{'loss': 0.732, 'grad_norm': 2.786039352416992, 'learning_rate': 4.737997054190397e-05, 'entropy': 0.9043587744235992, 'num_tokens': 398513.0, 'mean_token_accuracy': 0.7902904450893402, 'epoch': 2.67}
{'loss': 0.7359, 'grad_norm': 2.9016621112823486, 'learning_rate': 4.718609235648121e-05, 'entropy': 0.9737970381975174, 'num_tokens': 399120.0, 'mean_token_accuracy': 0.8041794449090958, 'epoch': 2.67}
{'loss': 0.6244, 'grad_norm': 3.288905143737793, 'learning_rate': 4.6992256604865333e-05, 'entropy': 0.7939340174198151, 'num_tokens': 399716.0, 'mean_token_accuracy': 0.8178980797529221, 'epoch': 2.68}
{'loss': 0.7615, 'grad_norm': 3.192544937133789, 'learning_rate': 4.6798466210104953e-05, 'entropy': 0.919306293129921, 'num_tokens': 400272.0, 'mean_token_accuracy': 0.8061705380678177, 'epoch': 2.69}
{'loss': 0.7252, 'grad_norm': 3.1661062240600586, 'learning_rate': 4.660472409456472e-05, 'entropy': 0.8950475603342056, 'num_tokens': 400811.0, 'mean_token_accuracy': 0.8055791854858398, 'epoch': 2.69}
{'loss': 0.5513, 'grad_norm': 2.806138277053833, 'learning_rate': 4.64110331798812e-05, 'entropy': 0.8073269575834274, 'num_tokens': 401335.0, 'mean_token_accuracy': 0.8423076868057251, 'epoch': 2.7}
{'loss': 0.6772, 'grad_norm': 3.5020060539245605, 'learning_rate': 4.621739638691887e-05, 'entropy': 0.9498953521251678, 'num_tokens': 401819.0, 'mean_token_accuracy': 0.8120239675045013, 'epoch': 2.7}
{'loss': 0.6228, 'grad_norm': 3.4239449501037598, 'learning_rate': 4.6023816635726066e-05, 'entropy': 0.8201316893100739, 'num_tokens': 402270.0, 'mean_token_accuracy': 0.8281460255384445, 'epoch': 2.71}
{'loss': 0.6147, 'grad_norm': 3.529266595840454, 'learning_rate': 4.583029684549089e-05, 'entropy': 0.7581432014703751, 'num_tokens': 402696.0, 'mean_token_accuracy': 0.8218429237604141, 'epoch': 2.72}
{'loss': 0.6132, 'grad_norm': 3.6258456707000732, 'learning_rate': 4.5636839934497274e-05, 'entropy': 0.7856215834617615, 'num_tokens': 403109.0, 'mean_token_accuracy': 0.8390538096427917, 'epoch': 2.72}
{'loss': 0.5604, 'grad_norm': 3.2864136695861816, 'learning_rate': 4.544344882008088e-05, 'entropy': 0.6760046184062958, 'num_tokens': 403485.0, 'mean_token_accuracy': 0.8598688691854477, 'epoch': 2.73}
{'loss': 0.6026, 'grad_norm': 4.191997051239014, 'learning_rate': 4.525012641858524e-05, 'entropy': 0.7535198777914047, 'num_tokens': 403827.0, 'mean_token_accuracy': 0.7990725338459015, 'epoch': 2.73}
{'loss': 0.4635, 'grad_norm': 3.783743381500244, 'learning_rate': 4.5056875645317564e-05, 'entropy': 0.6239349916577339, 'num_tokens': 404114.0, 'mean_token_accuracy': 0.8790618777275085, 'epoch': 2.74}
{'loss': 1.3601, 'grad_norm': 2.4902424812316895, 'learning_rate': 4.486369941450504e-05, 'entropy': 1.0692661851644516, 'num_tokens': 406133.0, 'mean_token_accuracy': 0.676251694560051, 'epoch': 2.75}
{'loss': 1.1249, 'grad_norm': 2.5415709018707275, 'learning_rate': 4.467060063925068e-05, 'entropy': 0.9689496606588364, 'num_tokens': 407832.0, 'mean_token_accuracy': 0.6836012899875641, 'epoch': 2.75}
{'loss': 0.9045, 'grad_norm': 2.420053005218506, 'learning_rate': 4.447758223148945e-05, 'entropy': 0.9057926535606384, 'num_tokens': 409296.0, 'mean_token_accuracy': 0.7686677128076553, 'epoch': 2.76}
{'loss': 0.8574, 'grad_norm': 2.679043769836426, 'learning_rate': 4.428464710194445e-05, 'entropy': 0.917514979839325, 'num_tokens': 410680.0, 'mean_token_accuracy': 0.7427781671285629, 'epoch': 2.76}
{'loss': 0.7781, 'grad_norm': 2.12701153755188, 'learning_rate': 4.4091798160082855e-05, 'entropy': 0.8457233905792236, 'num_tokens': 411997.0, 'mean_token_accuracy': 0.7745242714881897, 'epoch': 2.77}
{'loss': 0.4103, 'grad_norm': 1.5896433591842651, 'learning_rate': 4.3899038314072185e-05, 'entropy': 0.4494778849184513, 'num_tokens': 413291.0, 'mean_token_accuracy': 0.901751309633255, 'epoch': 2.78}
{'loss': 0.7427, 'grad_norm': 2.712264060974121, 'learning_rate': 4.3706370470736335e-05, 'entropy': 0.8109926134347916, 'num_tokens': 414526.0, 'mean_token_accuracy': 0.7893633842468262, 'epoch': 2.78}
{'loss': 0.8224, 'grad_norm': 2.521574020385742, 'learning_rate': 4.3513797535511876e-05, 'entropy': 0.853114664554596, 'num_tokens': 415739.0, 'mean_token_accuracy': 0.7762836664915085, 'epoch': 2.79}
{'loss': 1.0456, 'grad_norm': 2.298290729522705, 'learning_rate': 4.332132241240413e-05, 'entropy': 1.0503429174423218, 'num_tokens': 416907.0, 'mean_token_accuracy': 0.7179789990186691, 'epoch': 2.79}
{'loss': 1.0225, 'grad_norm': 2.609201669692993, 'learning_rate': 4.3128948003943336e-05, 'entropy': 1.087536409497261, 'num_tokens': 418035.0, 'mean_token_accuracy': 0.7350283861160278, 'epoch': 2.8}
{'loss': 0.8238, 'grad_norm': 2.4091796875, 'learning_rate': 4.2936677211141084e-05, 'entropy': 0.9903431236743927, 'num_tokens': 419110.0, 'mean_token_accuracy': 0.7822493016719818, 'epoch': 2.81}
{'loss': 0.6445, 'grad_norm': 2.41670298576355, 'learning_rate': 4.274451293344633e-05, 'entropy': 0.7239107117056847, 'num_tokens': 420131.0, 'mean_token_accuracy': 0.8103312253952026, 'epoch': 2.81}
{'loss': 0.8487, 'grad_norm': 2.996842384338379, 'learning_rate': 4.255245806870185e-05, 'entropy': 0.8470437005162239, 'num_tokens': 421114.0, 'mean_token_accuracy': 0.7821852266788483, 'epoch': 2.82}
{'loss': 0.9161, 'grad_norm': 2.6526122093200684, 'learning_rate': 4.236051551310038e-05, 'entropy': 0.9290631618350744, 'num_tokens': 422078.0, 'mean_token_accuracy': 0.7575088143348694, 'epoch': 2.82}
{'loss': 1.0023, 'grad_norm': 2.8416900634765625, 'learning_rate': 4.2168688161141176e-05, 'entropy': 1.07048861682415, 'num_tokens': 423022.0, 'mean_token_accuracy': 0.7510593235492706, 'epoch': 2.83}
{'loss': 1.1104, 'grad_norm': 2.9699795246124268, 'learning_rate': 4.1976978905586046e-05, 'entropy': 1.0982428938150406, 'num_tokens': 423932.0, 'mean_token_accuracy': 0.7167066484689713, 'epoch': 2.84}
{'loss': 0.8741, 'grad_norm': 2.5096702575683594, 'learning_rate': 4.178539063741604e-05, 'entropy': 0.9321097135543823, 'num_tokens': 424814.0, 'mean_token_accuracy': 0.7531167417764664, 'epoch': 2.84}
{'loss': 0.6957, 'grad_norm': 2.9687557220458984, 'learning_rate': 4.159392624578766e-05, 'entropy': 0.8116550296545029, 'num_tokens': 425670.0, 'mean_token_accuracy': 0.810882180929184, 'epoch': 2.85}
{'loss': 0.8363, 'grad_norm': 2.5391757488250732, 'learning_rate': 4.140258861798932e-05, 'entropy': 0.9292239844799042, 'num_tokens': 426502.0, 'mean_token_accuracy': 0.7632582187652588, 'epoch': 2.85}
{'loss': 0.7426, 'grad_norm': 2.551037549972534, 'learning_rate': 4.12113806393979e-05, 'entropy': 0.9182191491127014, 'num_tokens': 427309.0, 'mean_token_accuracy': 0.7870843410491943, 'epoch': 2.86}
{'loss': 0.9287, 'grad_norm': 2.9831948280334473, 'learning_rate': 4.1020305193435096e-05, 'entropy': 1.039366602897644, 'num_tokens': 428096.0, 'mean_token_accuracy': 0.7509520202875137, 'epoch': 2.87}
{'loss': 0.8871, 'grad_norm': 3.1111485958099365, 'learning_rate': 4.082936516152407e-05, 'entropy': 0.9830431640148163, 'num_tokens': 428871.0, 'mean_token_accuracy': 0.770714744925499, 'epoch': 2.87}
{'loss': 0.693, 'grad_norm': 3.020368814468384, 'learning_rate': 4.063856342304586e-05, 'entropy': 0.7693557888269424, 'num_tokens': 429620.0, 'mean_token_accuracy': 0.8100065439939499, 'epoch': 2.88}
{'loss': 0.7616, 'grad_norm': 2.4976000785827637, 'learning_rate': 4.0447902855296114e-05, 'entropy': 0.9103455692529678, 'num_tokens': 430347.0, 'mean_token_accuracy': 0.8144485801458359, 'epoch': 2.88}
{'loss': 0.912, 'grad_norm': 2.984355926513672, 'learning_rate': 4.025738633344161e-05, 'entropy': 1.144132822751999, 'num_tokens': 431045.0, 'mean_token_accuracy': 0.7636994123458862, 'epoch': 2.89}
{'loss': 0.6527, 'grad_norm': 2.6113038063049316, 'learning_rate': 4.006701673047683e-05, 'entropy': 0.7827958390116692, 'num_tokens': 431738.0, 'mean_token_accuracy': 0.818774476647377, 'epoch': 2.9}
{'loss': 0.9335, 'grad_norm': 2.9431183338165283, 'learning_rate': 3.9876796917180814e-05, 'entropy': 1.0933553129434586, 'num_tokens': 432418.0, 'mean_token_accuracy': 0.7544602453708649, 'epoch': 2.9}
{'loss': 0.7043, 'grad_norm': 2.520622730255127, 'learning_rate': 3.968672976207372e-05, 'entropy': 0.858560785651207, 'num_tokens': 433078.0, 'mean_token_accuracy': 0.793989285826683, 'epoch': 2.91}
{'loss': 0.7999, 'grad_norm': 2.90739369392395, 'learning_rate': 3.949681813137363e-05, 'entropy': 0.9613050073385239, 'num_tokens': 433712.0, 'mean_token_accuracy': 0.783120185136795, 'epoch': 2.91}
{'loss': 0.9686, 'grad_norm': 3.031651735305786, 'learning_rate': 3.930706488895328e-05, 'entropy': 1.0598332583904266, 'num_tokens': 434324.0, 'mean_token_accuracy': 0.7644551545381546, 'epoch': 2.92}
{'loss': 0.8242, 'grad_norm': 3.5617239475250244, 'learning_rate': 3.911747289629697e-05, 'entropy': 1.0118118822574615, 'num_tokens': 434917.0, 'mean_token_accuracy': 0.7804290503263474, 'epoch': 2.93}
{'loss': 0.7205, 'grad_norm': 3.084350109100342, 'learning_rate': 3.8928045012457314e-05, 'entropy': 1.010154977440834, 'num_tokens': 435489.0, 'mean_token_accuracy': 0.7811827063560486, 'epoch': 2.93}
{'loss': 0.726, 'grad_norm': 3.474627733230591, 'learning_rate': 3.873878409401212e-05, 'entropy': 0.9193386435508728, 'num_tokens': 436044.0, 'mean_token_accuracy': 0.7985580712556839, 'epoch': 2.94}
{'loss': 0.5483, 'grad_norm': 2.9347665309906006, 'learning_rate': 3.8549692995021454e-05, 'entropy': 0.7842485904693604, 'num_tokens': 436551.0, 'mean_token_accuracy': 0.8566230684518814, 'epoch': 2.94}
{'loss': 0.7884, 'grad_norm': 3.9292404651641846, 'learning_rate': 3.836077456698437e-05, 'entropy': 0.9535427242517471, 'num_tokens': 437034.0, 'mean_token_accuracy': 0.7949533760547638, 'epoch': 2.95}
{'loss': 0.6244, 'grad_norm': 3.420846462249756, 'learning_rate': 3.817203165879616e-05, 'entropy': 0.8936704695224762, 'num_tokens': 437499.0, 'mean_token_accuracy': 0.832964763045311, 'epoch': 2.96}
{'loss': 0.5253, 'grad_norm': 3.381687641143799, 'learning_rate': 3.79834671167052e-05, 'entropy': 0.7335586398839951, 'num_tokens': 437954.0, 'mean_token_accuracy': 0.8469253033399582, 'epoch': 2.96}
{'loss': 0.5581, 'grad_norm': 3.7389893531799316, 'learning_rate': 3.779508378427011e-05, 'entropy': 0.8195018768310547, 'num_tokens': 438389.0, 'mean_token_accuracy': 0.8606895804405212, 'epoch': 2.97}
{'loss': 0.3729, 'grad_norm': 3.182722806930542, 'learning_rate': 3.760688450231691e-05, 'entropy': 0.5515269264578819, 'num_tokens': 438797.0, 'mean_token_accuracy': 0.9079660922288895, 'epoch': 2.97}
{'loss': 0.6398, 'grad_norm': 4.270784854888916, 'learning_rate': 3.741887210889605e-05, 'entropy': 0.8628223463892937, 'num_tokens': 439131.0, 'mean_token_accuracy': 0.8390999734401703, 'epoch': 2.98}
{'loss': 0.5289, 'grad_norm': 4.020496845245361, 'learning_rate': 3.723104943923979e-05, 'entropy': 0.6973675489425659, 'num_tokens': 439440.0, 'mean_token_accuracy': 0.8638642877340317, 'epoch': 2.99}
{'loss': 0.9306, 'grad_norm': 1.907652497291565, 'learning_rate': 3.704341932571925e-05, 'entropy': 0.9138283878564835, 'num_tokens': 440882.0, 'mean_token_accuracy': 0.7390443533658981, 'epoch': 2.99}
{'loss': 0.8601, 'grad_norm': 3.379054546356201, 'learning_rate': 3.68559845978019e-05, 'entropy': 0.8444960862398148, 'num_tokens': 441597.0, 'mean_token_accuracy': 0.7662162631750107, 'epoch': 3.0}
{'loss': 0.4719, 'grad_norm': 5.865952014923096, 'learning_rate': 3.666874808200871e-05, 'entropy': 0.7190922498703003, 'num_tokens': 441705.0, 'mean_token_accuracy': 0.8971962332725525, 'epoch': 3.0}
{'loss': 0.9911, 'grad_norm': 1.6665441989898682, 'learning_rate': 3.6481712601871654e-05, 'entropy': 0.9776846617460251, 'num_tokens': 444364.0, 'mean_token_accuracy': 0.7343210130929947, 'epoch': 3.01}
{'loss': 0.7538, 'grad_norm': 1.95341956615448, 'learning_rate': 3.6294880977891036e-05, 'entropy': 0.8872155398130417, 'num_tokens': 446242.0, 'mean_token_accuracy': 0.7827666848897934, 'epoch': 3.01}
{'loss': 0.5909, 'grad_norm': 1.6800414323806763, 'learning_rate': 3.610825602749307e-05, 'entropy': 0.6451631411910057, 'num_tokens': 447890.0, 'mean_token_accuracy': 0.8251261711120605, 'epoch': 3.02}
{'loss': 0.5326, 'grad_norm': 1.977118968963623, 'learning_rate': 3.592184056498727e-05, 'entropy': 0.7049839347600937, 'num_tokens': 449311.0, 'mean_token_accuracy': 0.8516039848327637, 'epoch': 3.02}
{'loss': 0.424, 'grad_norm': 1.7269376516342163, 'learning_rate': 3.573563740152407e-05, 'entropy': 0.6168644428253174, 'num_tokens': 450619.0, 'mean_token_accuracy': 0.8820157051086426, 'epoch': 3.03}
{'loss': 0.6683, 'grad_norm': 2.1234331130981445, 'learning_rate': 3.554964934505249e-05, 'entropy': 0.8519486784934998, 'num_tokens': 451885.0, 'mean_token_accuracy': 0.8087327033281326, 'epoch': 3.04}
{'loss': 0.5463, 'grad_norm': 2.2918474674224854, 'learning_rate': 3.536387920027762e-05, 'entropy': 0.7130813971161842, 'num_tokens': 453116.0, 'mean_token_accuracy': 0.8477758467197418, 'epoch': 3.04}
{'loss': 0.6446, 'grad_norm': 2.4477243423461914, 'learning_rate': 3.517832976861855e-05, 'entropy': 0.7316697835922241, 'num_tokens': 454301.0, 'mean_token_accuracy': 0.8153292387723923, 'epoch': 3.05}
{'loss': 0.6597, 'grad_norm': 2.7085368633270264, 'learning_rate': 3.4993003848165935e-05, 'entropy': 0.7749504297971725, 'num_tokens': 455439.0, 'mean_token_accuracy': 0.8114258199930191, 'epoch': 3.05}
{'loss': 0.5533, 'grad_norm': 2.8625879287719727, 'learning_rate': 3.48079042336399e-05, 'entropy': 0.6118142940104008, 'num_tokens': 456501.0, 'mean_token_accuracy': 0.8433092534542084, 'epoch': 3.06}
{'loss': 0.4284, 'grad_norm': 2.3768911361694336, 'learning_rate': 3.462303371634789e-05, 'entropy': 0.6187149360775948, 'num_tokens': 457549.0, 'mean_token_accuracy': 0.8719894587993622, 'epoch': 3.07}
{'loss': 0.5467, 'grad_norm': 2.1754512786865234, 'learning_rate': 3.443839508414251e-05, 'entropy': 0.6118019670248032, 'num_tokens': 458566.0, 'mean_token_accuracy': 0.8680598735809326, 'epoch': 3.07}
{'loss': 0.5841, 'grad_norm': 3.1181788444519043, 'learning_rate': 3.425399112137958e-05, 'entropy': 0.6793028116226196, 'num_tokens': 459545.0, 'mean_token_accuracy': 0.8398348540067673, 'epoch': 3.08}
{'loss': 0.4388, 'grad_norm': 2.325342893600464, 'learning_rate': 3.406982460887606e-05, 'entropy': 0.5734735652804375, 'num_tokens': 460489.0, 'mean_token_accuracy': 0.8828247487545013, 'epoch': 3.08}
{'loss': 0.717, 'grad_norm': 3.417091131210327, 'learning_rate': 3.388589832386818e-05, 'entropy': 0.8783266693353653, 'num_tokens': 461396.0, 'mean_token_accuracy': 0.804095908999443, 'epoch': 3.09}
{'loss': 0.583, 'grad_norm': 3.2314138412475586, 'learning_rate': 3.370221503996951e-05, 'entropy': 0.7602724581956863, 'num_tokens': 462272.0, 'mean_token_accuracy': 0.8350543081760406, 'epoch': 3.1}
{'loss': 0.1609, 'grad_norm': 1.3558244705200195, 'learning_rate': 3.351877752712917e-05, 'entropy': 0.2398744598031044, 'num_tokens': 463126.0, 'mean_token_accuracy': 0.9517721682786942, 'epoch': 3.1}
{'loss': 0.5649, 'grad_norm': 3.239386796951294, 'learning_rate': 3.333558855159007e-05, 'entropy': 0.7030050754547119, 'num_tokens': 463969.0, 'mean_token_accuracy': 0.8498329818248749, 'epoch': 3.11}
{'loss': 0.4804, 'grad_norm': 3.0771255493164062, 'learning_rate': 3.315265087584711e-05, 'entropy': 0.6164320334792137, 'num_tokens': 464786.0, 'mean_token_accuracy': 0.8736090511083603, 'epoch': 3.11}
{'loss': 0.345, 'grad_norm': 3.017984628677368, 'learning_rate': 3.2969967258605645e-05, 'entropy': 0.5390076115727425, 'num_tokens': 465566.0, 'mean_token_accuracy': 0.8920350670814514, 'epoch': 3.12}
{'loss': 0.553, 'grad_norm': 3.0421769618988037, 'learning_rate': 3.278754045473975e-05, 'entropy': 0.6928874179720879, 'num_tokens': 466323.0, 'mean_token_accuracy': 0.8509211093187332, 'epoch': 3.13}
{'loss': 0.4196, 'grad_norm': 2.1435933113098145, 'learning_rate': 3.2605373215250856e-05, 'entropy': 0.5546650215983391, 'num_tokens': 467063.0, 'mean_token_accuracy': 0.8901060074567795, 'epoch': 3.13}
{'loss': 0.4683, 'grad_norm': 3.2343814373016357, 'learning_rate': 3.242346828722602e-05, 'entropy': 0.6562196165323257, 'num_tokens': 467784.0, 'mean_token_accuracy': 0.8630690574645996, 'epoch': 3.14}
{'loss': 0.4411, 'grad_norm': 3.0660347938537598, 'learning_rate': 3.224182841379674e-05, 'entropy': 0.6468003988265991, 'num_tokens': 468486.0, 'mean_token_accuracy': 0.8827153146266937, 'epoch': 3.14}
{'loss': 0.679, 'grad_norm': 3.0837395191192627, 'learning_rate': 3.2060456334097444e-05, 'entropy': 0.8597269803285599, 'num_tokens': 469175.0, 'mean_token_accuracy': 0.828277587890625, 'epoch': 3.15}
{'loss': 0.4196, 'grad_norm': 2.5129048824310303, 'learning_rate': 3.18793547832242e-05, 'entropy': 0.6109993681311607, 'num_tokens': 469848.0, 'mean_token_accuracy': 0.8925865739583969, 'epoch': 3.16}
{'loss': 0.3848, 'grad_norm': 4.004247665405273, 'learning_rate': 3.169852649219357e-05, 'entropy': 0.47975043207407, 'num_tokens': 470505.0, 'mean_token_accuracy': 0.883489266037941, 'epoch': 3.16}
{'loss': 0.2845, 'grad_norm': 2.823509454727173, 'learning_rate': 3.151797418790126e-05, 'entropy': 0.5023976936936378, 'num_tokens': 471146.0, 'mean_token_accuracy': 0.9340282082557678, 'epoch': 3.17}
{'loss': 0.4137, 'grad_norm': 2.687821388244629, 'learning_rate': 3.133770059308114e-05, 'entropy': 0.5860616639256477, 'num_tokens': 471758.0, 'mean_token_accuracy': 0.8803223669528961, 'epoch': 3.17}
{'loss': 0.3844, 'grad_norm': 3.2021327018737793, 'learning_rate': 3.1157708426264096e-05, 'entropy': 0.5576566755771637, 'num_tokens': 472352.0, 'mean_token_accuracy': 0.9050101786851883, 'epoch': 3.18}
{'loss': 0.3245, 'grad_norm': 2.623994827270508, 'learning_rate': 3.0978000401737136e-05, 'entropy': 0.5227561369538307, 'num_tokens': 472932.0, 'mean_token_accuracy': 0.913179948925972, 'epoch': 3.19}
{'loss': 0.3918, 'grad_norm': 3.626624584197998, 'learning_rate': 3.0798579229502347e-05, 'entropy': 0.6111031249165535, 'num_tokens': 473497.0, 'mean_token_accuracy': 0.8947519659996033, 'epoch': 3.19}
{'loss': 0.4086, 'grad_norm': 3.2897727489471436, 'learning_rate': 3.0619447615236054e-05, 'entropy': 0.499600313603878, 'num_tokens': 474049.0, 'mean_token_accuracy': 0.8903364986181259, 'epoch': 3.2}
{'loss': 0.2588, 'grad_norm': 3.167681932449341, 'learning_rate': 3.0440608260248105e-05, 'entropy': 0.4004208445549011, 'num_tokens': 474566.0, 'mean_token_accuracy': 0.9145432710647583, 'epoch': 3.2}
{'loss': 0.3219, 'grad_norm': 3.1589133739471436, 'learning_rate': 3.0262063861441003e-05, 'entropy': 0.5224428623914719, 'num_tokens': 475056.0, 'mean_token_accuracy': 0.9238924384117126, 'epoch': 3.21}
{'loss': 0.3372, 'grad_norm': 3.8212201595306396, 'learning_rate': 3.0083817111269356e-05, 'entropy': 0.5731977671384811, 'num_tokens': 475536.0, 'mean_token_accuracy': 0.9138613939285278, 'epoch': 3.22}
{'loss': 0.2707, 'grad_norm': 3.3275675773620605, 'learning_rate': 2.9905870697699174e-05, 'entropy': 0.4453834369778633, 'num_tokens': 475985.0, 'mean_token_accuracy': 0.9390132278203964, 'epoch': 3.22}
{'loss': 0.1481, 'grad_norm': 2.2890682220458984, 'learning_rate': 2.9728227304167456e-05, 'entropy': 0.28299740701913834, 'num_tokens': 476382.0, 'mean_token_accuracy': 0.9695655405521393, 'epoch': 3.23}
{'loss': 0.232, 'grad_norm': 2.897038221359253, 'learning_rate': 2.9550889609541543e-05, 'entropy': 0.3214118331670761, 'num_tokens': 476737.0, 'mean_token_accuracy': 0.9466077387332916, 'epoch': 3.23}
{'loss': 0.2033, 'grad_norm': 3.5736706256866455, 'learning_rate': 2.9373860288078925e-05, 'entropy': 0.3564777672290802, 'num_tokens': 477069.0, 'mean_token_accuracy': 0.9541362673044205, 'epoch': 3.24}
{'loss': 0.163, 'grad_norm': 3.471498727798462, 'learning_rate': 2.919714200938678e-05, 'entropy': 0.26301486417651176, 'num_tokens': 477377.0, 'mean_token_accuracy': 0.9537767469882965, 'epoch': 3.25}
{'loss': 0.795, 'grad_norm': 2.1750502586364746, 'learning_rate': 2.9020737438381718e-05, 'entropy': 0.7452477514743805, 'num_tokens': 479810.0, 'mean_token_accuracy': 0.7670690268278122, 'epoch': 3.25}
{'loss': 0.9405, 'grad_norm': 2.711998462677002, 'learning_rate': 2.884464923524971e-05, 'entropy': 0.7959445714950562, 'num_tokens': 481726.0, 'mean_token_accuracy': 0.7654158473014832, 'epoch': 3.26}
{'loss': 0.6233, 'grad_norm': 2.542558431625366, 'learning_rate': 2.866888005540579e-05, 'entropy': 0.6118560284376144, 'num_tokens': 483359.0, 'mean_token_accuracy': 0.8168193101882935, 'epoch': 3.26}
{'loss': 0.6412, 'grad_norm': 2.1245200634002686, 'learning_rate': 2.8493432549454215e-05, 'entropy': 0.6704689115285873, 'num_tokens': 484844.0, 'mean_token_accuracy': 0.8345956355333328, 'epoch': 3.27}
{'loss': 0.4929, 'grad_norm': 2.5748918056488037, 'learning_rate': 2.831830936314831e-05, 'entropy': 0.618374664336443, 'num_tokens': 486176.0, 'mean_token_accuracy': 0.8577637523412704, 'epoch': 3.28}
{'loss': 0.3742, 'grad_norm': 2.251333713531494, 'learning_rate': 2.8143513137350663e-05, 'entropy': 0.507083248347044, 'num_tokens': 487453.0, 'mean_token_accuracy': 0.8937539011240005, 'epoch': 3.28}
{'loss': 0.6746, 'grad_norm': 3.0774121284484863, 'learning_rate': 2.7969046507993345e-05, 'entropy': 0.6994407474994659, 'num_tokens': 488655.0, 'mean_token_accuracy': 0.8212157785892487, 'epoch': 3.29}
{'loss': 0.6402, 'grad_norm': 2.7775919437408447, 'learning_rate': 2.7794912106038025e-05, 'entropy': 0.7611866742372513, 'num_tokens': 489827.0, 'mean_token_accuracy': 0.8220152258872986, 'epoch': 3.29}
{'loss': 0.7552, 'grad_norm': 3.3261632919311523, 'learning_rate': 2.7621112557436447e-05, 'entropy': 0.8109378218650818, 'num_tokens': 490970.0, 'mean_token_accuracy': 0.7929151654243469, 'epoch': 3.3}
{'loss': 0.7373, 'grad_norm': 2.809173107147217, 'learning_rate': 2.744765048309068e-05, 'entropy': 0.8084109127521515, 'num_tokens': 492079.0, 'mean_token_accuracy': 0.8155835717916489, 'epoch': 3.31}
{'loss': 0.6221, 'grad_norm': 2.7310476303100586, 'learning_rate': 2.7274528498813763e-05, 'entropy': 0.717153787612915, 'num_tokens': 493134.0, 'mean_token_accuracy': 0.8296157270669937, 'epoch': 3.31}
{'loss': 0.6099, 'grad_norm': 2.833718776702881, 'learning_rate': 2.7101749215290085e-05, 'entropy': 0.7084047719836235, 'num_tokens': 494152.0, 'mean_token_accuracy': 0.8239213526248932, 'epoch': 3.32}
{'loss': 0.502, 'grad_norm': 3.0827019214630127, 'learning_rate': 2.6929315238036124e-05, 'entropy': 0.5838570147752762, 'num_tokens': 495143.0, 'mean_token_accuracy': 0.8571113795042038, 'epoch': 3.32}
{'loss': 0.6254, 'grad_norm': 3.0255191326141357, 'learning_rate': 2.6757229167361165e-05, 'entropy': 0.7389193624258041, 'num_tokens': 496112.0, 'mean_token_accuracy': 0.8339454978704453, 'epoch': 3.33}
{'loss': 0.4831, 'grad_norm': 2.815382957458496, 'learning_rate': 2.6585493598327983e-05, 'entropy': 0.5923565849661827, 'num_tokens': 497037.0, 'mean_token_accuracy': 0.8515633940696716, 'epoch': 3.34}
{'loss': 0.3493, 'grad_norm': 2.424442768096924, 'learning_rate': 2.6414111120713862e-05, 'entropy': 0.47401778399944305, 'num_tokens': 497943.0, 'mean_token_accuracy': 0.9023478180170059, 'epoch': 3.34}
{'loss': 0.638, 'grad_norm': 2.844930410385132, 'learning_rate': 2.6243084318971394e-05, 'entropy': 0.6995582431554794, 'num_tokens': 498836.0, 'mean_token_accuracy': 0.8325152546167374, 'epoch': 3.35}
{'loss': 0.4437, 'grad_norm': 3.0303292274475098, 'learning_rate': 2.607241577218959e-05, 'entropy': 0.6400220394134521, 'num_tokens': 499711.0, 'mean_token_accuracy': 0.8806283622980118, 'epoch': 3.35}
{'loss': 0.3743, 'grad_norm': 2.453225612640381, 'learning_rate': 2.590210805405494e-05, 'entropy': 0.5533977523446083, 'num_tokens': 500567.0, 'mean_token_accuracy': 0.8963959068059921, 'epoch': 3.36}
{'loss': 0.4482, 'grad_norm': 2.69026780128479, 'learning_rate': 2.5732163732812675e-05, 'entropy': 0.5636066906154156, 'num_tokens': 501402.0, 'mean_token_accuracy': 0.8750014305114746, 'epoch': 3.37}
{'loss': 0.5595, 'grad_norm': 3.1737658977508545, 'learning_rate': 2.556258537122799e-05, 'entropy': 0.7294699996709824, 'num_tokens': 502215.0, 'mean_token_accuracy': 0.8424942940473557, 'epoch': 3.37}
{'loss': 0.6266, 'grad_norm': 3.519002676010132, 'learning_rate': 2.5393375526547327e-05, 'entropy': 0.7412671148777008, 'num_tokens': 503006.0, 'mean_token_accuracy': 0.8398302048444748, 'epoch': 3.38}
{'loss': 0.6812, 'grad_norm': 3.2982614040374756, 'learning_rate': 2.522453675045998e-05, 'entropy': 0.7908658534288406, 'num_tokens': 503774.0, 'mean_token_accuracy': 0.8132672756910324, 'epoch': 3.38}
{'loss': 0.5389, 'grad_norm': 3.1742284297943115, 'learning_rate': 2.5056071589059447e-05, 'entropy': 0.6965954601764679, 'num_tokens': 504509.0, 'mean_token_accuracy': 0.8643774092197418, 'epoch': 3.39}
{'loss': 0.5582, 'grad_norm': 3.363048553466797, 'learning_rate': 2.4887982582805126e-05, 'entropy': 0.6794351041316986, 'num_tokens': 505212.0, 'mean_token_accuracy': 0.8485935628414154, 'epoch': 3.4}
{'loss': 0.4154, 'grad_norm': 3.3362817764282227, 'learning_rate': 2.4720272266483973e-05, 'entropy': 0.6126924008131027, 'num_tokens': 505896.0, 'mean_token_accuracy': 0.8809148073196411, 'epoch': 3.4}
{'loss': 0.3665, 'grad_norm': 2.7795650959014893, 'learning_rate': 2.455294316917232e-05, 'entropy': 0.5367417559027672, 'num_tokens': 506566.0, 'mean_token_accuracy': 0.894857868552208, 'epoch': 3.41}
{'loss': 0.4753, 'grad_norm': 3.810314655303955, 'learning_rate': 2.438599781419771e-05, 'entropy': 0.658558651804924, 'num_tokens': 507220.0, 'mean_token_accuracy': 0.8586805760860443, 'epoch': 3.42}
{'loss': 0.3132, 'grad_norm': 2.9055869579315186, 'learning_rate': 2.421943871910078e-05, 'entropy': 0.46452417224645615, 'num_tokens': 507858.0, 'mean_token_accuracy': 0.9038053900003433, 'epoch': 3.42}
{'loss': 0.364, 'grad_norm': 2.7965593338012695, 'learning_rate': 2.405326839559745e-05, 'entropy': 0.5247919484972954, 'num_tokens': 508472.0, 'mean_token_accuracy': 0.9050676673650742, 'epoch': 3.43}
{'loss': 0.4052, 'grad_norm': 2.6950018405914307, 'learning_rate': 2.3887489349540832e-05, 'entropy': 0.6024319604039192, 'num_tokens': 509078.0, 'mean_token_accuracy': 0.8886092752218246, 'epoch': 3.43}
{'loss': 0.3985, 'grad_norm': 3.449427843093872, 'learning_rate': 2.372210408088368e-05, 'entropy': 0.48741503804922104, 'num_tokens': 509666.0, 'mean_token_accuracy': 0.8937015533447266, 'epoch': 3.44}
{'loss': 0.4072, 'grad_norm': 3.9040958881378174, 'learning_rate': 2.355711508364049e-05, 'entropy': 0.5986730083823204, 'num_tokens': 510222.0, 'mean_token_accuracy': 0.8914322108030319, 'epoch': 3.45}
{'loss': 0.2789, 'grad_norm': 3.3579699993133545, 'learning_rate': 2.339252484585005e-05, 'entropy': 0.4676615744829178, 'num_tokens': 510761.0, 'mean_token_accuracy': 0.9255902767181396, 'epoch': 3.45}
{'loss': 0.3468, 'grad_norm': 3.0554683208465576, 'learning_rate': 2.3228335849537764e-05, 'entropy': 0.5288742408156395, 'num_tokens': 511281.0, 'mean_token_accuracy': 0.9264872372150421, 'epoch': 3.46}
{'loss': 0.3626, 'grad_norm': 3.7783255577087402, 'learning_rate': 2.3064550570678378e-05, 'entropy': 0.49661942571401596, 'num_tokens': 511774.0, 'mean_token_accuracy': 0.8973688632249832, 'epoch': 3.46}
{'loss': 0.2958, 'grad_norm': 3.428905487060547, 'learning_rate': 2.2901171479158513e-05, 'entropy': 0.46772998571395874, 'num_tokens': 512242.0, 'mean_token_accuracy': 0.9072348773479462, 'epoch': 3.47}
{'loss': 0.2577, 'grad_norm': 3.289055585861206, 'learning_rate': 2.273820103873947e-05, 'entropy': 0.38315456360578537, 'num_tokens': 512678.0, 'mean_token_accuracy': 0.9282844662666321, 'epoch': 3.48}
{'loss': 0.212, 'grad_norm': 2.8359177112579346, 'learning_rate': 2.2575641707020146e-05, 'entropy': 0.32204336673021317, 'num_tokens': 513090.0, 'mean_token_accuracy': 0.9555471837520599, 'epoch': 3.48}
{'loss': 0.1838, 'grad_norm': 2.905301094055176, 'learning_rate': 2.241349593539982e-05, 'entropy': 0.3278392478823662, 'num_tokens': 513437.0, 'mean_token_accuracy': 0.9560663849115372, 'epoch': 3.49}
{'loss': 0.1846, 'grad_norm': 3.2202413082122803, 'learning_rate': 2.2251766169041355e-05, 'entropy': 0.32172831147909164, 'num_tokens': 513726.0, 'mean_token_accuracy': 0.9616257101297379, 'epoch': 3.49}
{'loss': 0.7531, 'grad_norm': 2.0003573894500732, 'learning_rate': 2.2090454846834164e-05, 'entropy': 0.7306284755468369, 'num_tokens': 516605.0, 'mean_token_accuracy': 0.784162849187851, 'epoch': 3.5}
{'loss': 0.8336, 'grad_norm': 2.2009177207946777, 'learning_rate': 2.1929564401357593e-05, 'entropy': 0.7835957258939743, 'num_tokens': 518717.0, 'mean_token_accuracy': 0.7712904512882233, 'epoch': 3.51}
{'loss': 0.7586, 'grad_norm': 2.298870086669922, 'learning_rate': 2.1769097258844083e-05, 'entropy': 0.739078938961029, 'num_tokens': 520532.0, 'mean_token_accuracy': 0.7888057082891464, 'epoch': 3.51}
{'loss': 0.6008, 'grad_norm': 2.4752395153045654, 'learning_rate': 2.1609055839142683e-05, 'entropy': 0.6989565268158913, 'num_tokens': 521998.0, 'mean_token_accuracy': 0.8233090192079544, 'epoch': 3.52}
{'loss': 0.5196, 'grad_norm': 2.5329928398132324, 'learning_rate': 2.144944255568255e-05, 'entropy': 0.6144474893808365, 'num_tokens': 523372.0, 'mean_token_accuracy': 0.8415521085262299, 'epoch': 3.52}
{'loss': 0.5601, 'grad_norm': 2.521439790725708, 'learning_rate': 2.1290259815436487e-05, 'entropy': 0.6086113378405571, 'num_tokens': 524663.0, 'mean_token_accuracy': 0.8425430953502655, 'epoch': 3.53}
{'loss': 0.7071, 'grad_norm': 2.851331949234009, 'learning_rate': 2.1131510018884755e-05, 'entropy': 0.8049023896455765, 'num_tokens': 525894.0, 'mean_token_accuracy': 0.7911256104707718, 'epoch': 3.54}
{'loss': 0.4758, 'grad_norm': 2.6606931686401367, 'learning_rate': 2.0973195559978757e-05, 'entropy': 0.529242217540741, 'num_tokens': 527078.0, 'mean_token_accuracy': 0.8611825257539749, 'epoch': 3.54}
{'loss': 0.4676, 'grad_norm': 3.0359110832214355, 'learning_rate': 2.0815318826105002e-05, 'entropy': 0.5412887334823608, 'num_tokens': 528229.0, 'mean_token_accuracy': 0.8656870275735855, 'epoch': 3.55}
{'loss': 0.5802, 'grad_norm': 3.143653631210327, 'learning_rate': 2.0657882198049132e-05, 'entropy': 0.6474462300539017, 'num_tokens': 529349.0, 'mean_token_accuracy': 0.8405875265598297, 'epoch': 3.55}
{'loss': 0.5571, 'grad_norm': 3.1203455924987793, 'learning_rate': 2.050088804995992e-05, 'entropy': 0.6996991485357285, 'num_tokens': 530425.0, 'mean_token_accuracy': 0.8471455276012421, 'epoch': 3.56}
{'loss': 0.3312, 'grad_norm': 2.727443218231201, 'learning_rate': 2.0344338749313597e-05, 'entropy': 0.4242370277643204, 'num_tokens': 531472.0, 'mean_token_accuracy': 0.9145451188087463, 'epoch': 3.57}
{'loss': 0.5496, 'grad_norm': 3.1846365928649902, 'learning_rate': 2.018823665687801e-05, 'entropy': 0.6503195911645889, 'num_tokens': 532492.0, 'mean_token_accuracy': 0.8456084877252579, 'epoch': 3.57}
{'loss': 0.7244, 'grad_norm': 3.308702230453491, 'learning_rate': 2.0032584126677167e-05, 'entropy': 0.7997982054948807, 'num_tokens': 533478.0, 'mean_token_accuracy': 0.8014238178730011, 'epoch': 3.58}
{'loss': 0.4297, 'grad_norm': 2.6630730628967285, 'learning_rate': 1.9877383505955606e-05, 'entropy': 0.5046481490135193, 'num_tokens': 534434.0, 'mean_token_accuracy': 0.8812389075756073, 'epoch': 3.58}
{'loss': 0.5124, 'grad_norm': 2.6541836261749268, 'learning_rate': 1.9722637135143068e-05, 'entropy': 0.6632211059331894, 'num_tokens': 535380.0, 'mean_token_accuracy': 0.8460331708192825, 'epoch': 3.59}
{'loss': 0.4666, 'grad_norm': 2.8708536624908447, 'learning_rate': 1.9568347347819232e-05, 'entropy': 0.5955108515918255, 'num_tokens': 536298.0, 'mean_token_accuracy': 0.8668500334024429, 'epoch': 3.6}
{'loss': 0.5376, 'grad_norm': 3.160964250564575, 'learning_rate': 1.9414516470678423e-05, 'entropy': 0.6145008057355881, 'num_tokens': 537166.0, 'mean_token_accuracy': 0.8443918228149414, 'epoch': 3.6}
{'loss': 0.3752, 'grad_norm': 2.636265993118286, 'learning_rate': 1.9261146823494657e-05, 'entropy': 0.5052944421768188, 'num_tokens': 538008.0, 'mean_token_accuracy': 0.8962225317955017, 'epoch': 3.61}
{'loss': 0.7055, 'grad_norm': 3.2322561740875244, 'learning_rate': 1.9108240719086517e-05, 'entropy': 0.8075186759233475, 'num_tokens': 538824.0, 'mean_token_accuracy': 0.8105297684669495, 'epoch': 3.61}
{'loss': 0.4785, 'grad_norm': 3.0376689434051514, 'learning_rate': 1.895580046328244e-05, 'entropy': 0.6030717268586159, 'num_tokens': 539619.0, 'mean_token_accuracy': 0.8824686855077744, 'epoch': 3.62}
{'loss': 0.4962, 'grad_norm': 3.4955387115478516, 'learning_rate': 1.880382835488572e-05, 'entropy': 0.6121270880103111, 'num_tokens': 540385.0, 'mean_token_accuracy': 0.8578123301267624, 'epoch': 3.63}
{'loss': 0.3928, 'grad_norm': 2.9689934253692627, 'learning_rate': 1.865232668564009e-05, 'entropy': 0.5295829251408577, 'num_tokens': 541084.0, 'mean_token_accuracy': 0.884834498167038, 'epoch': 3.63}
{'loss': 0.4615, 'grad_norm': 2.984346628189087, 'learning_rate': 1.8501297740195035e-05, 'entropy': 0.5587364807724953, 'num_tokens': 541765.0, 'mean_token_accuracy': 0.8712473958730698, 'epoch': 3.64}
{'loss': 0.4076, 'grad_norm': 3.1841726303100586, 'learning_rate': 1.8350743796071272e-05, 'entropy': 0.5613399222493172, 'num_tokens': 542422.0, 'mean_token_accuracy': 0.8770648539066315, 'epoch': 3.64}
{'loss': 0.4528, 'grad_norm': 3.0851473808288574, 'learning_rate': 1.8200667123626593e-05, 'entropy': 0.617046907544136, 'num_tokens': 543057.0, 'mean_token_accuracy': 0.8810632824897766, 'epoch': 3.65}
{'loss': 0.3667, 'grad_norm': 3.237126588821411, 'learning_rate': 1.805106998602143e-05, 'entropy': 0.5030144304037094, 'num_tokens': 543674.0, 'mean_token_accuracy': 0.8858737647533417, 'epoch': 3.66}
{'loss': 0.5889, 'grad_norm': 3.4194533824920654, 'learning_rate': 1.790195463918486e-05, 'entropy': 0.705841101706028, 'num_tokens': 544271.0, 'mean_token_accuracy': 0.8569442480802536, 'epoch': 3.66}
{'loss': 0.4233, 'grad_norm': 3.963669538497925, 'learning_rate': 1.775332333178052e-05, 'entropy': 0.5559355095028877, 'num_tokens': 544858.0, 'mean_token_accuracy': 0.8813991248607635, 'epoch': 3.67}
{'loss': 0.3858, 'grad_norm': 3.4605963230133057, 'learning_rate': 1.7605178305172752e-05, 'entropy': 0.6094141006469727, 'num_tokens': 545430.0, 'mean_token_accuracy': 0.897646963596344, 'epoch': 3.67}
{'loss': 0.3109, 'grad_norm': 3.067781448364258, 'learning_rate': 1.7457521793392727e-05, 'entropy': 0.5303854718804359, 'num_tokens': 545993.0, 'mean_token_accuracy': 0.9123072922229767, 'epoch': 3.68}
{'loss': 0.3283, 'grad_norm': 3.3113861083984375, 'learning_rate': 1.731035602310483e-05, 'entropy': 0.46804051101207733, 'num_tokens': 546545.0, 'mean_token_accuracy': 0.8978977650403976, 'epoch': 3.69}
{'loss': 0.2667, 'grad_norm': 2.940455198287964, 'learning_rate': 1.7163683213573062e-05, 'entropy': 0.4454641044139862, 'num_tokens': 547071.0, 'mean_token_accuracy': 0.9273528307676315, 'epoch': 3.69}
{'loss': 0.3761, 'grad_norm': 3.7057461738586426, 'learning_rate': 1.7017505576627523e-05, 'entropy': 0.5530801191926003, 'num_tokens': 547575.0, 'mean_token_accuracy': 0.896143227815628, 'epoch': 3.7}
{'loss': 0.3134, 'grad_norm': 3.9584829807281494, 'learning_rate': 1.6871825316631112e-05, 'entropy': 0.484640933573246, 'num_tokens': 548052.0, 'mean_token_accuracy': 0.9176517426967621, 'epoch': 3.7}
{'loss': 0.2547, 'grad_norm': 2.7798845767974854, 'learning_rate': 1.6726644630446257e-05, 'entropy': 0.44830964505672455, 'num_tokens': 548517.0, 'mean_token_accuracy': 0.9436262100934982, 'epoch': 3.71}
{'loss': 0.2361, 'grad_norm': 2.8759665489196777, 'learning_rate': 1.658196570740185e-05, 'entropy': 0.3417629115283489, 'num_tokens': 548960.0, 'mean_token_accuracy': 0.9453275799751282, 'epoch': 3.72}
{'loss': 0.2068, 'grad_norm': 2.8041226863861084, 'learning_rate': 1.6437790729260104e-05, 'entropy': 0.4039744362235069, 'num_tokens': 549391.0, 'mean_token_accuracy': 0.9555580466985703, 'epoch': 3.72}
{'loss': 0.2419, 'grad_norm': 3.234995126724243, 'learning_rate': 1.6294121870183792e-05, 'entropy': 0.37162578850984573, 'num_tokens': 549770.0, 'mean_token_accuracy': 0.9477694928646088, 'epoch': 3.73}
{'loss': 0.2421, 'grad_norm': 3.550931215286255, 'learning_rate': 1.61509612967034e-05, 'entropy': 0.40136051923036575, 'num_tokens': 550107.0, 'mean_token_accuracy': 0.9308304637670517, 'epoch': 3.73}
{'loss': 0.251, 'grad_norm': 3.113515615463257, 'learning_rate': 1.6008311167684365e-05, 'entropy': 0.36450138315558434, 'num_tokens': 550419.0, 'mean_token_accuracy': 0.9491081684827805, 'epoch': 3.74}
{'loss': 0.7627, 'grad_norm': 2.011570930480957, 'learning_rate': 1.5866173634294717e-05, 'entropy': 0.8291130810976028, 'num_tokens': 552535.0, 'mean_token_accuracy': 0.7713960260152817, 'epoch': 3.75}
{'loss': 0.7712, 'grad_norm': 2.345663547515869, 'learning_rate': 1.572455083997245e-05, 'entropy': 0.7873587012290955, 'num_tokens': 554415.0, 'mean_token_accuracy': 0.7788004726171494, 'epoch': 3.75}
{'loss': 0.7794, 'grad_norm': 2.158132791519165, 'learning_rate': 1.5583444920393347e-05, 'entropy': 0.8681664019823074, 'num_tokens': 556144.0, 'mean_token_accuracy': 0.7969674170017242, 'epoch': 3.76}
{'loss': 0.5552, 'grad_norm': 1.9821239709854126, 'learning_rate': 1.5442858003438628e-05, 'entropy': 0.6452053934335709, 'num_tokens': 557770.0, 'mean_token_accuracy': 0.8376172631978989, 'epoch': 3.76}
{'loss': 0.4129, 'grad_norm': 2.1020965576171875, 'learning_rate': 1.5302792209163002e-05, 'entropy': 0.5255088433623314, 'num_tokens': 559255.0, 'mean_token_accuracy': 0.8828621953725815, 'epoch': 3.77}
{'loss': 0.5492, 'grad_norm': 2.9118542671203613, 'learning_rate': 1.5163249649762612e-05, 'entropy': 0.6173962652683258, 'num_tokens': 560655.0, 'mean_token_accuracy': 0.8484081327915192, 'epoch': 3.78}
{'loss': 0.4161, 'grad_norm': 2.6388282775878906, 'learning_rate': 1.5024232429543184e-05, 'entropy': 0.5279993116855621, 'num_tokens': 562003.0, 'mean_token_accuracy': 0.8668438643217087, 'epoch': 3.78}
{'loss': 0.3419, 'grad_norm': 2.0471878051757812, 'learning_rate': 1.488574264488835e-05, 'entropy': 0.43730996176600456, 'num_tokens': 563313.0, 'mean_token_accuracy': 0.904249981045723, 'epoch': 3.79}
{'loss': 0.7758, 'grad_norm': 3.268556833267212, 'learning_rate': 1.4747782384227959e-05, 'entropy': 0.8084235489368439, 'num_tokens': 564572.0, 'mean_token_accuracy': 0.7878672033548355, 'epoch': 3.79}
{'loss': 0.5972, 'grad_norm': 3.264294147491455, 'learning_rate': 1.4610353728006676e-05, 'entropy': 0.633698582649231, 'num_tokens': 565786.0, 'mean_token_accuracy': 0.8257583826780319, 'epoch': 3.8}
{'loss': 0.6568, 'grad_norm': 3.053370952606201, 'learning_rate': 1.4473458748652496e-05, 'entropy': 0.6735861003398895, 'num_tokens': 566931.0, 'mean_token_accuracy': 0.8161063194274902, 'epoch': 3.81}
{'loss': 0.6261, 'grad_norm': 2.9938807487487793, 'learning_rate': 1.433709951054561e-05, 'entropy': 0.7280328497290611, 'num_tokens': 568029.0, 'mean_token_accuracy': 0.8308831006288528, 'epoch': 3.81}
{'loss': 0.5469, 'grad_norm': 2.857224225997925, 'learning_rate': 1.4201278069987167e-05, 'entropy': 0.5885571762919426, 'num_tokens': 569077.0, 'mean_token_accuracy': 0.8622836321592331, 'epoch': 3.82}
{'loss': 0.3483, 'grad_norm': 2.098095178604126, 'learning_rate': 1.4065996475168325e-05, 'entropy': 0.3784006666392088, 'num_tokens': 570058.0, 'mean_token_accuracy': 0.9107470959424973, 'epoch': 3.82}
{'loss': 0.8202, 'grad_norm': 3.8138654232025146, 'learning_rate': 1.393125676613939e-05, 'entropy': 0.838669627904892, 'num_tokens': 571002.0, 'mean_token_accuracy': 0.781291738152504, 'epoch': 3.83}
{'loss': 0.564, 'grad_norm': 2.8679606914520264, 'learning_rate': 1.3797060974778952e-05, 'entropy': 0.6094313710927963, 'num_tokens': 571913.0, 'mean_token_accuracy': 0.8596331626176834, 'epoch': 3.84}
{'loss': 0.2669, 'grad_norm': 2.39431095123291, 'learning_rate': 1.3663411124763381e-05, 'entropy': 0.2986500356346369, 'num_tokens': 572798.0, 'mean_token_accuracy': 0.9267513602972031, 'epoch': 3.84}
{'loss': 0.4536, 'grad_norm': 2.822474241256714, 'learning_rate': 1.3530309231536176e-05, 'entropy': 0.5309649743139744, 'num_tokens': 573651.0, 'mean_token_accuracy': 0.8658483922481537, 'epoch': 3.85}
{'loss': 0.5125, 'grad_norm': 2.9014029502868652, 'learning_rate': 1.3397757302277647e-05, 'entropy': 0.601752258837223, 'num_tokens': 574465.0, 'mean_token_accuracy': 0.8678897321224213, 'epoch': 3.85}
{'loss': 0.4076, 'grad_norm': 3.3599345684051514, 'learning_rate': 1.3265757335874635e-05, 'entropy': 0.5167762413620949, 'num_tokens': 575273.0, 'mean_token_accuracy': 0.874378114938736, 'epoch': 3.86}
{'loss': 0.3764, 'grad_norm': 3.178285837173462, 'learning_rate': 1.313431132289038e-05, 'entropy': 0.4902461916208267, 'num_tokens': 576070.0, 'mean_token_accuracy': 0.8926411718130112, 'epoch': 3.87}
{'loss': 0.4671, 'grad_norm': 2.8065757751464844, 'learning_rate': 1.3003421245534486e-05, 'entropy': 0.5606269910931587, 'num_tokens': 576846.0, 'mean_token_accuracy': 0.8912280350923538, 'epoch': 3.87}
{'loss': 0.3265, 'grad_norm': 3.007335662841797, 'learning_rate': 1.2873089077632994e-05, 'entropy': 0.3999204933643341, 'num_tokens': 577585.0, 'mean_token_accuracy': 0.9211776107549667, 'epoch': 3.88}
{'loss': 0.6109, 'grad_norm': 3.638834238052368, 'learning_rate': 1.2743316784598719e-05, 'entropy': 0.7287621796131134, 'num_tokens': 578294.0, 'mean_token_accuracy': 0.8577728718519211, 'epoch': 3.88}
{'loss': 0.3123, 'grad_norm': 2.781665802001953, 'learning_rate': 1.2614106323401492e-05, 'entropy': 0.408865574747324, 'num_tokens': 578965.0, 'mean_token_accuracy': 0.9113543480634689, 'epoch': 3.89}
{'loss': 0.3762, 'grad_norm': 3.1036088466644287, 'learning_rate': 1.2485459642538727e-05, 'entropy': 0.5347893461585045, 'num_tokens': 579611.0, 'mean_token_accuracy': 0.8894280195236206, 'epoch': 3.9}
{'loss': 0.3647, 'grad_norm': 3.3184473514556885, 'learning_rate': 1.2357378682005994e-05, 'entropy': 0.45575476437807083, 'num_tokens': 580232.0, 'mean_token_accuracy': 0.8995372653007507, 'epoch': 3.9}
{'loss': 0.429, 'grad_norm': 3.8002641201019287, 'learning_rate': 1.2229865373267818e-05, 'entropy': 0.5591019541025162, 'num_tokens': 580846.0, 'mean_token_accuracy': 0.878803625702858, 'epoch': 3.91}
{'loss': 0.3162, 'grad_norm': 3.0319457054138184, 'learning_rate': 1.2102921639228521e-05, 'entropy': 0.45895277708768845, 'num_tokens': 581449.0, 'mean_token_accuracy': 0.913255050778389, 'epoch': 3.91}
{'loss': 0.3321, 'grad_norm': 3.367048740386963, 'learning_rate': 1.1976549394203167e-05, 'entropy': 0.4320603087544441, 'num_tokens': 582046.0, 'mean_token_accuracy': 0.9072699248790741, 'epoch': 3.92}
{'loss': 0.3872, 'grad_norm': 3.7182912826538086, 'learning_rate': 1.1850750543888834e-05, 'entropy': 0.5494477599859238, 'num_tokens': 582614.0, 'mean_token_accuracy': 0.9004706144332886, 'epoch': 3.93}
{'loss': 0.3027, 'grad_norm': 2.76407790184021, 'learning_rate': 1.1725526985335688e-05, 'entropy': 0.5391190052032471, 'num_tokens': 583154.0, 'mean_token_accuracy': 0.932778462767601, 'epoch': 3.93}
{'loss': 0.2565, 'grad_norm': 2.9714648723602295, 'learning_rate': 1.160088060691858e-05, 'entropy': 0.381986778229475, 'num_tokens': 583682.0, 'mean_token_accuracy': 0.9315666854381561, 'epoch': 3.94}
{'loss': 0.2245, 'grad_norm': 2.4579689502716064, 'learning_rate': 1.1476813288308386e-05, 'entropy': 0.3865225799381733, 'num_tokens': 584200.0, 'mean_token_accuracy': 0.9473046660423279, 'epoch': 3.94}
{'loss': 0.2603, 'grad_norm': 2.9895310401916504, 'learning_rate': 1.1353326900443805e-05, 'entropy': 0.42640551179647446, 'num_tokens': 584691.0, 'mean_token_accuracy': 0.9426165074110031, 'epoch': 3.95}
{'loss': 0.2355, 'grad_norm': 3.246157646179199, 'learning_rate': 1.1230423305503068e-05, 'entropy': 0.3855155408382416, 'num_tokens': 585148.0, 'mean_token_accuracy': 0.9360042661428452, 'epoch': 3.96}
{'loss': 0.1988, 'grad_norm': 3.2552709579467773, 'learning_rate': 1.1108104356875854e-05, 'entropy': 0.34484202414751053, 'num_tokens': 585582.0, 'mean_token_accuracy': 0.9395928531885147, 'epoch': 3.96}
{'loss': 0.2725, 'grad_norm': 3.6848511695861816, 'learning_rate': 1.0986371899135366e-05, 'entropy': 0.3953104503452778, 'num_tokens': 586002.0, 'mean_token_accuracy': 0.9444728791713715, 'epoch': 3.97}
{'loss': 0.1751, 'grad_norm': 3.224548101425171, 'learning_rate': 1.08652277680105e-05, 'entropy': 0.31807342916727066, 'num_tokens': 586406.0, 'mean_token_accuracy': 0.9650624543428421, 'epoch': 3.97}
{'loss': 0.1974, 'grad_norm': 3.0707409381866455, 'learning_rate': 1.0744673790358207e-05, 'entropy': 0.35876449197530746, 'num_tokens': 586759.0, 'mean_token_accuracy': 0.950776681303978, 'epoch': 3.98}
{'loss': 0.2395, 'grad_norm': 2.4697072505950928, 'learning_rate': 1.0624711784135848e-05, 'entropy': 0.30703747645020485, 'num_tokens': 587029.0, 'mean_token_accuracy': 0.9573822468519211, 'epoch': 3.99}
{'loss': 0.4898, 'grad_norm': 2.494044542312622, 'learning_rate': 1.0505343558373897e-05, 'entropy': 0.4714055508375168, 'num_tokens': 588120.0, 'mean_token_accuracy': 0.8742078840732574, 'epoch': 3.99}
{'loss': 0.5651, 'grad_norm': 3.787045478820801, 'learning_rate': 1.0386570913148586e-05, 'entropy': 0.7085917145013809, 'num_tokens': 588832.0, 'mean_token_accuracy': 0.8508399426937103, 'epoch': 4.0}
{'loss': 0.1373, 'grad_norm': 2.309964895248413, 'learning_rate': 1.0268395639554745e-05, 'entropy': 0.23186752200126648, 'num_tokens': 588940.0, 'mean_token_accuracy': 0.9719626307487488, 'epoch': 4.0}
{'loss': 0.5315, 'grad_norm': 1.7594962120056152, 'learning_rate': 1.015081951967885e-05, 'entropy': 0.5853341817855835, 'num_tokens': 591324.0, 'mean_token_accuracy': 0.8628797233104706, 'epoch': 4.01}
{'loss': 0.6352, 'grad_norm': 2.2652032375335693, 'learning_rate': 1.00338443265721e-05, 'entropy': 0.7317667454481125, 'num_tokens': 593111.0, 'mean_token_accuracy': 0.8237095028162003, 'epoch': 4.01}
{'loss': 0.3677, 'grad_norm': 1.558659553527832, 'learning_rate': 9.91747182422374e-06, 'entropy': 0.5390277206897736, 'num_tokens': 594722.0, 'mean_token_accuracy': 0.9044066816568375, 'epoch': 4.02}
{'loss': 0.2833, 'grad_norm': 1.9867154359817505, 'learning_rate': 9.801703767534382e-06, 'entropy': 0.43622986972332, 'num_tokens': 596060.0, 'mean_token_accuracy': 0.9167031794786453, 'epoch': 4.02}
{'loss': 0.2482, 'grad_norm': 1.8478666543960571, 'learning_rate': 9.686541902289609e-06, 'entropy': 0.40638208389282227, 'num_tokens': 597365.0, 'mean_token_accuracy': 0.931511640548706, 'epoch': 4.03}
{'loss': 0.4086, 'grad_norm': 2.273909330368042, 'learning_rate': 9.571987965133604e-06, 'entropy': 0.5521983504295349, 'num_tokens': 598651.0, 'mean_token_accuracy': 0.889907032251358, 'epoch': 4.04}
{'loss': 0.4643, 'grad_norm': 2.537520170211792, 'learning_rate': 9.458043683542967e-06, 'entropy': 0.6678812503814697, 'num_tokens': 599920.0, 'mean_token_accuracy': 0.8741960823535919, 'epoch': 4.04}
{'loss': 0.4178, 'grad_norm': 2.4615607261657715, 'learning_rate': 9.344710775800708e-06, 'entropy': 0.5997335463762283, 'num_tokens': 601130.0, 'mean_token_accuracy': 0.8903651386499405, 'epoch': 4.05}
{'loss': 0.6114, 'grad_norm': 2.6020708084106445, 'learning_rate': 9.23199095097026e-06, 'entropy': 0.7814756259322166, 'num_tokens': 602247.0, 'mean_token_accuracy': 0.841380387544632, 'epoch': 4.05}
{'loss': 0.2002, 'grad_norm': 1.9111164808273315, 'learning_rate': 9.119885908869803e-06, 'entropy': 0.33663464337587357, 'num_tokens': 603297.0, 'mean_token_accuracy': 0.958864763379097, 'epoch': 4.06}
{'loss': 0.5644, 'grad_norm': 2.6420459747314453, 'learning_rate': 9.00839734004651e-06, 'entropy': 0.6843406409025192, 'num_tokens': 604309.0, 'mean_token_accuracy': 0.8430664390325546, 'epoch': 4.07}
{'loss': 0.3142, 'grad_norm': 2.515836238861084, 'learning_rate': 8.897526925751177e-06, 'entropy': 0.4758957624435425, 'num_tokens': 605280.0, 'mean_token_accuracy': 0.9151050001382828, 'epoch': 4.07}
{'loss': 0.4355, 'grad_norm': 2.7816736698150635, 'learning_rate': 8.787276337912787e-06, 'entropy': 0.6025324687361717, 'num_tokens': 606232.0, 'mean_token_accuracy': 0.8943931311368942, 'epoch': 4.08}
{'loss': 0.196, 'grad_norm': 1.6704262495040894, 'learning_rate': 8.677647239113296e-06, 'entropy': 0.29021377861499786, 'num_tokens': 607163.0, 'mean_token_accuracy': 0.942684605717659, 'epoch': 4.08}
{'loss': 0.3751, 'grad_norm': 2.6141860485076904, 'learning_rate': 8.568641282562656e-06, 'entropy': 0.5161142125725746, 'num_tokens': 608059.0, 'mean_token_accuracy': 0.9014438390731812, 'epoch': 4.09}
{'loss': 0.3882, 'grad_norm': 2.4758379459381104, 'learning_rate': 8.460260112073743e-06, 'entropy': 0.48444121517241, 'num_tokens': 608925.0, 'mean_token_accuracy': 0.9067195653915405, 'epoch': 4.1}
{'loss': 0.1919, 'grad_norm': 2.019479751586914, 'learning_rate': 8.3525053620377e-06, 'entropy': 0.3304008115082979, 'num_tokens': 609775.0, 'mean_token_accuracy': 0.9515386372804642, 'epoch': 4.1}
{'loss': 0.3472, 'grad_norm': 2.943664789199829, 'learning_rate': 8.24537865739919e-06, 'entropy': 0.5154951587319374, 'num_tokens': 610607.0, 'mean_token_accuracy': 0.9106099158525467, 'epoch': 4.11}
{'loss': 0.3731, 'grad_norm': 3.2787246704101562, 'learning_rate': 8.13888161363197e-06, 'entropy': 0.49667898565530777, 'num_tokens': 611407.0, 'mean_token_accuracy': 0.9080867171287537, 'epoch': 4.11}
{'loss': 0.3285, 'grad_norm': 2.8266091346740723, 'learning_rate': 8.033015836714441e-06, 'entropy': 0.48954421281814575, 'num_tokens': 612181.0, 'mean_token_accuracy': 0.9235076606273651, 'epoch': 4.12}
{'loss': 0.2267, 'grad_norm': 3.0015406608581543, 'learning_rate': 7.927782923105525e-06, 'entropy': 0.3954590782523155, 'num_tokens': 612941.0, 'mean_token_accuracy': 0.9457284361124039, 'epoch': 4.13}
{'loss': 0.1846, 'grad_norm': 2.1861257553100586, 'learning_rate': 7.823184459720562e-06, 'entropy': 0.31494300439953804, 'num_tokens': 613672.0, 'mean_token_accuracy': 0.9569299817085266, 'epoch': 4.13}
{'loss': 0.2643, 'grad_norm': 2.7057077884674072, 'learning_rate': 7.719222023907307e-06, 'entropy': 0.4047349914908409, 'num_tokens': 614380.0, 'mean_token_accuracy': 0.9361656159162521, 'epoch': 4.14}
{'loss': 0.2754, 'grad_norm': 2.573031187057495, 'learning_rate': 7.61589718342226e-06, 'entropy': 0.5074405446648598, 'num_tokens': 615073.0, 'mean_token_accuracy': 0.9261349141597748, 'epoch': 4.14}
{'loss': 0.2378, 'grad_norm': 3.3585045337677, 'learning_rate': 7.5132114964069245e-06, 'entropy': 0.3618439696729183, 'num_tokens': 615734.0, 'mean_token_accuracy': 0.9300454556941986, 'epoch': 4.15}
{'loss': 0.2467, 'grad_norm': 3.017808437347412, 'learning_rate': 7.41116651136437e-06, 'entropy': 0.38286320865154266, 'num_tokens': 616377.0, 'mean_token_accuracy': 0.9484264701604843, 'epoch': 4.16}
{'loss': 0.1902, 'grad_norm': 2.4597673416137695, 'learning_rate': 7.309763767135841e-06, 'entropy': 0.3099515847861767, 'num_tokens': 616994.0, 'mean_token_accuracy': 0.9543672055006027, 'epoch': 4.16}
{'loss': 0.2172, 'grad_norm': 2.812849998474121, 'learning_rate': 7.20900479287761e-06, 'entropy': 0.31743836402893066, 'num_tokens': 617593.0, 'mean_token_accuracy': 0.9360860884189606, 'epoch': 4.17}
{'loss': 0.1862, 'grad_norm': 2.8577380180358887, 'learning_rate': 7.1088911080378625e-06, 'entropy': 0.34042292460799217, 'num_tokens': 618166.0, 'mean_token_accuracy': 0.9490723162889481, 'epoch': 4.17}
{'loss': 0.2032, 'grad_norm': 3.080730438232422, 'learning_rate': 7.009424222333788e-06, 'entropy': 0.3649933338165283, 'num_tokens': 618726.0, 'mean_token_accuracy': 0.9443323016166687, 'epoch': 4.18}
{'loss': 0.1787, 'grad_norm': 2.5698533058166504, 'learning_rate': 6.910605635728839e-06, 'entropy': 0.28387368470430374, 'num_tokens': 619266.0, 'mean_token_accuracy': 0.964587464928627, 'epoch': 4.19}
{'loss': 0.0928, 'grad_norm': 1.4439756870269775, 'learning_rate': 6.812436838410108e-06, 'entropy': 0.19450674578547478, 'num_tokens': 619790.0, 'mean_token_accuracy': 0.984615370631218, 'epoch': 4.19}
{'loss': 0.1326, 'grad_norm': 2.220367670059204, 'learning_rate': 6.714919310765827e-06, 'entropy': 0.28068165108561516, 'num_tokens': 620287.0, 'mean_token_accuracy': 0.969621866941452, 'epoch': 4.2}
{'loss': 0.1417, 'grad_norm': 3.037346839904785, 'learning_rate': 6.618054523363054e-06, 'entropy': 0.24006013572216034, 'num_tokens': 620772.0, 'mean_token_accuracy': 0.9625344276428223, 'epoch': 4.2}
{'loss': 0.1064, 'grad_norm': 1.8329106569290161, 'learning_rate': 6.521843936925548e-06, 'entropy': 0.20841609686613083, 'num_tokens': 621236.0, 'mean_token_accuracy': 0.9737181216478348, 'epoch': 4.21}
{'loss': 0.1319, 'grad_norm': 2.0686588287353516, 'learning_rate': 6.4262890023116805e-06, 'entropy': 0.21980871818959713, 'num_tokens': 621680.0, 'mean_token_accuracy': 0.9705137014389038, 'epoch': 4.22}
{'loss': 0.1036, 'grad_norm': 1.7303745746612549, 'learning_rate': 6.33139116049255e-06, 'entropy': 0.19683555886149406, 'num_tokens': 622108.0, 'mean_token_accuracy': 0.9716048538684845, 'epoch': 4.22}
{'loss': 0.1273, 'grad_norm': 3.0594234466552734, 'learning_rate': 6.2371518425303085e-06, 'entropy': 0.20102280750870705, 'num_tokens': 622518.0, 'mean_token_accuracy': 0.9680606424808502, 'epoch': 4.23}
{'loss': 0.0994, 'grad_norm': 1.3834487199783325, 'learning_rate': 6.143572469556524e-06, 'entropy': 0.16334390826523304, 'num_tokens': 622887.0, 'mean_token_accuracy': 0.9777901917695999, 'epoch': 4.23}
{'loss': 0.1195, 'grad_norm': 1.7197844982147217, 'learning_rate': 6.050654452750804e-06, 'entropy': 0.23207945004105568, 'num_tokens': 623222.0, 'mean_token_accuracy': 0.9728522151708603, 'epoch': 4.24}
{'loss': 0.1061, 'grad_norm': 1.6313834190368652, 'learning_rate': 5.958399193319453e-06, 'entropy': 0.16843418404459953, 'num_tokens': 623497.0, 'mean_token_accuracy': 0.9675789624452591, 'epoch': 4.25}
{'loss': 0.6409, 'grad_norm': 2.725202798843384, 'learning_rate': 5.866808082474407e-06, 'entropy': 0.6547714322805405, 'num_tokens': 626038.0, 'mean_token_accuracy': 0.8108907490968704, 'epoch': 4.25}
{'loss': 0.7619, 'grad_norm': 2.3258097171783447, 'learning_rate': 5.775882501412216e-06, 'entropy': 0.7491233646869659, 'num_tokens': 628079.0, 'mean_token_accuracy': 0.7843362540006638, 'epoch': 4.26}
{'loss': 0.7632, 'grad_norm': 2.8747620582580566, 'learning_rate': 5.6856238212932065e-06, 'entropy': 0.7475741505622864, 'num_tokens': 629892.0, 'mean_token_accuracy': 0.7927161604166031, 'epoch': 4.26}
{'loss': 0.3771, 'grad_norm': 2.329710006713867, 'learning_rate': 5.5960334032208296e-06, 'entropy': 0.3998899608850479, 'num_tokens': 631562.0, 'mean_token_accuracy': 0.891809269785881, 'epoch': 4.27}
{'loss': 0.608, 'grad_norm': 2.620527982711792, 'learning_rate': 5.507112598221109e-06, 'entropy': 0.6467855498194695, 'num_tokens': 633072.0, 'mean_token_accuracy': 0.8391246795654297, 'epoch': 4.28}
{'loss': 0.4043, 'grad_norm': 3.2567477226257324, 'learning_rate': 5.418862747222325e-06, 'entropy': 0.534157820045948, 'num_tokens': 634494.0, 'mean_token_accuracy': 0.8788330107927322, 'epoch': 4.28}
{'loss': 0.3386, 'grad_norm': 2.673638105392456, 'learning_rate': 5.331285181034707e-06, 'entropy': 0.3965189531445503, 'num_tokens': 635831.0, 'mean_token_accuracy': 0.915527880191803, 'epoch': 4.29}
{'loss': 0.4108, 'grad_norm': 3.1899895668029785, 'learning_rate': 5.244381220330452e-06, 'entropy': 0.5145158171653748, 'num_tokens': 637072.0, 'mean_token_accuracy': 0.8703092932701111, 'epoch': 4.29}
{'loss': 0.3526, 'grad_norm': 3.4268686771392822, 'learning_rate': 5.15815217562372e-06, 'entropy': 0.4703240171074867, 'num_tokens': 638245.0, 'mean_token_accuracy': 0.8943379521369934, 'epoch': 4.3}
{'loss': 0.4726, 'grad_norm': 2.9543581008911133, 'learning_rate': 5.072599347250978e-06, 'entropy': 0.5400343611836433, 'num_tokens': 639376.0, 'mean_token_accuracy': 0.8730590343475342, 'epoch': 4.31}
{'loss': 0.4612, 'grad_norm': 3.1042237281799316, 'learning_rate': 4.9877240253512755e-06, 'entropy': 0.5613406002521515, 'num_tokens': 640479.0, 'mean_token_accuracy': 0.8877730071544647, 'epoch': 4.31}
{'loss': 0.2401, 'grad_norm': 2.362255334854126, 'learning_rate': 4.903527489846865e-06, 'entropy': 0.30256155505776405, 'num_tokens': 641534.0, 'mean_token_accuracy': 0.9335821121931076, 'epoch': 4.32}
{'loss': 0.2189, 'grad_norm': 2.191554307937622, 'learning_rate': 4.820011010423897e-06, 'entropy': 0.28399568423628807, 'num_tokens': 642567.0, 'mean_token_accuracy': 0.9406606554985046, 'epoch': 4.32}
{'loss': 0.3989, 'grad_norm': 3.0910863876342773, 'learning_rate': 4.737175846513237e-06, 'entropy': 0.4641919955611229, 'num_tokens': 643569.0, 'mean_token_accuracy': 0.9010103940963745, 'epoch': 4.33}
{'loss': 0.2116, 'grad_norm': 1.8484739065170288, 'learning_rate': 4.655023247271512e-06, 'entropy': 0.270965326577425, 'num_tokens': 644528.0, 'mean_token_accuracy': 0.9420296549797058, 'epoch': 4.34}
{'loss': 0.2257, 'grad_norm': 2.0791385173797607, 'learning_rate': 4.573554451562234e-06, 'entropy': 0.3201003819704056, 'num_tokens': 645450.0, 'mean_token_accuracy': 0.934779092669487, 'epoch': 4.34}
{'loss': 0.4192, 'grad_norm': 2.847743272781372, 'learning_rate': 4.4927706879371366e-06, 'entropy': 0.5398269966244698, 'num_tokens': 646351.0, 'mean_token_accuracy': 0.898599237203598, 'epoch': 4.35}
{'loss': 0.2368, 'grad_norm': 2.3159267902374268, 'learning_rate': 4.412673174617682e-06, 'entropy': 0.31965070590376854, 'num_tokens': 647238.0, 'mean_token_accuracy': 0.9344180971384048, 'epoch': 4.35}
{'loss': 0.1403, 'grad_norm': 1.6173115968704224, 'learning_rate': 4.3332631194766214e-06, 'entropy': 0.2237703762948513, 'num_tokens': 648101.0, 'mean_token_accuracy': 0.9674511253833771, 'epoch': 4.36}
{'loss': 0.2057, 'grad_norm': 2.094482183456421, 'learning_rate': 4.2545417200198445e-06, 'entropy': 0.3063201420009136, 'num_tokens': 648948.0, 'mean_token_accuracy': 0.9453168660402298, 'epoch': 4.37}
{'loss': 0.2683, 'grad_norm': 2.402271270751953, 'learning_rate': 4.176510163368275e-06, 'entropy': 0.38460133969783783, 'num_tokens': 649777.0, 'mean_token_accuracy': 0.934148833155632, 'epoch': 4.37}
{'loss': 0.3685, 'grad_norm': 2.8064541816711426, 'learning_rate': 4.099169626240001e-06, 'entropy': 0.45320650190114975, 'num_tokens': 650583.0, 'mean_token_accuracy': 0.9026850759983063, 'epoch': 4.38}
{'loss': 0.4075, 'grad_norm': 3.2120087146759033, 'learning_rate': 4.022521274932506e-06, 'entropy': 0.5678321793675423, 'num_tokens': 651362.0, 'mean_token_accuracy': 0.8729972988367081, 'epoch': 4.38}
{'loss': 0.3209, 'grad_norm': 3.127983808517456, 'learning_rate': 3.946566265305091e-06, 'entropy': 0.45589151978492737, 'num_tokens': 652093.0, 'mean_token_accuracy': 0.9070752710103989, 'epoch': 4.39}
{'loss': 0.2933, 'grad_norm': 2.5401723384857178, 'learning_rate': 3.871305742761466e-06, 'entropy': 0.4928859919309616, 'num_tokens': 652778.0, 'mean_token_accuracy': 0.9285530000925064, 'epoch': 4.4}
{'loss': 0.2118, 'grad_norm': 2.2846601009368896, 'learning_rate': 3.7967408422324213e-06, 'entropy': 0.30242393910884857, 'num_tokens': 653435.0, 'mean_token_accuracy': 0.9463836848735809, 'epoch': 4.4}
{'loss': 0.2318, 'grad_norm': 2.6520729064941406, 'learning_rate': 3.7228726881587907e-06, 'entropy': 0.3661445379257202, 'num_tokens': 654084.0, 'mean_token_accuracy': 0.9441365301609039, 'epoch': 4.41}
{'loss': 0.1945, 'grad_norm': 2.3831188678741455, 'learning_rate': 3.649702394474397e-06, 'entropy': 0.3037320002913475, 'num_tokens': 654724.0, 'mean_token_accuracy': 0.948132112622261, 'epoch': 4.42}
{'loss': 0.2549, 'grad_norm': 2.547105073928833, 'learning_rate': 3.5772310645893815e-06, 'entropy': 0.34283867478370667, 'num_tokens': 655346.0, 'mean_token_accuracy': 0.9352324903011322, 'epoch': 4.42}
{'loss': 0.2271, 'grad_norm': 2.7832674980163574, 'learning_rate': 3.5054597913734e-06, 'entropy': 0.37051786482334137, 'num_tokens': 655955.0, 'mean_token_accuracy': 0.9375481456518173, 'epoch': 4.43}
{'loss': 0.4111, 'grad_norm': 2.7530975341796875, 'learning_rate': 3.4343896571393142e-06, 'entropy': 0.5114892162382603, 'num_tokens': 656553.0, 'mean_token_accuracy': 0.912386417388916, 'epoch': 4.43}
{'loss': 0.2547, 'grad_norm': 3.1607470512390137, 'learning_rate': 3.3640217336267543e-06, 'entropy': 0.44515521079301834, 'num_tokens': 657137.0, 'mean_token_accuracy': 0.9431486874818802, 'epoch': 4.44}
{'loss': 0.2494, 'grad_norm': 2.966811180114746, 'learning_rate': 3.294357081985988e-06, 'entropy': 0.39981649816036224, 'num_tokens': 657692.0, 'mean_token_accuracy': 0.9456215053796768, 'epoch': 4.45}
{'loss': 0.1793, 'grad_norm': 2.2339744567871094, 'learning_rate': 3.22539675276195e-06, 'entropy': 0.35689493268728256, 'num_tokens': 658231.0, 'mean_token_accuracy': 0.9625456780195236, 'epoch': 4.45}
{'loss': 0.1215, 'grad_norm': 1.6018235683441162, 'learning_rate': 3.1571417858783538e-06, 'entropy': 0.2190820537507534, 'num_tokens': 658758.0, 'mean_token_accuracy': 0.9751301407814026, 'epoch': 4.46}
{'loss': 0.1829, 'grad_norm': 2.434441566467285, 'learning_rate': 3.089593210622016e-06, 'entropy': 0.2954721413552761, 'num_tokens': 659270.0, 'mean_token_accuracy': 0.9566193073987961, 'epoch': 4.46}
{'loss': 0.1456, 'grad_norm': 2.2442877292633057, 'learning_rate': 3.0227520456273694e-06, 'entropy': 0.24251527711749077, 'num_tokens': 659741.0, 'mean_token_accuracy': 0.9678323864936829, 'epoch': 4.47}
{'loss': 0.1242, 'grad_norm': 1.8465290069580078, 'learning_rate': 2.9566192988610684e-06, 'entropy': 0.26697104796767235, 'num_tokens': 660173.0, 'mean_token_accuracy': 0.9719494432210922, 'epoch': 4.48}
{'loss': 0.1116, 'grad_norm': 1.303348183631897, 'learning_rate': 2.89119596760683e-06, 'entropy': 0.21264832839369774, 'num_tokens': 660568.0, 'mean_token_accuracy': 0.9798143953084946, 'epoch': 4.48}
{'loss': 0.117, 'grad_norm': 1.883695363998413, 'learning_rate': 2.826483038450306e-06, 'entropy': 0.2064109817147255, 'num_tokens': 660913.0, 'mean_token_accuracy': 0.9675963073968887, 'epoch': 4.49}
{'loss': 0.11, 'grad_norm': 2.652390956878662, 'learning_rate': 2.762481487264329e-06, 'entropy': 0.17305468395352364, 'num_tokens': 661206.0, 'mean_token_accuracy': 0.9657799005508423, 'epoch': 4.49}
{'loss': 0.682, 'grad_norm': 2.0060999393463135, 'learning_rate': 2.6991922791940904e-06, 'entropy': 0.7073216587305069, 'num_tokens': 663802.0, 'mean_token_accuracy': 0.8133124858140945, 'epoch': 4.5}
{'loss': 0.851, 'grad_norm': 2.4535226821899414, 'learning_rate': 2.636616368642636e-06, 'entropy': 0.8997239917516708, 'num_tokens': 665681.0, 'mean_token_accuracy': 0.7636360973119736, 'epoch': 4.51}
{'loss': 0.5122, 'grad_norm': 2.3323566913604736, 'learning_rate': 2.5747546992564595e-06, 'entropy': 0.6026121899485588, 'num_tokens': 667318.0, 'mean_token_accuracy': 0.8519444912672043, 'epoch': 4.51}
{'loss': 0.3111, 'grad_norm': 2.2023978233337402, 'learning_rate': 2.5136082039112996e-06, 'entropy': 0.43349945172667503, 'num_tokens': 668823.0, 'mean_token_accuracy': 0.9140069037675858, 'epoch': 4.52}
{'loss': 0.2651, 'grad_norm': 2.291414737701416, 'learning_rate': 2.4531778046980314e-06, 'entropy': 0.41250310465693474, 'num_tokens': 670233.0, 'mean_token_accuracy': 0.923583909869194, 'epoch': 4.52}
{'loss': 0.4689, 'grad_norm': 2.6295101642608643, 'learning_rate': 2.3934644129087856e-06, 'entropy': 0.5666822493076324, 'num_tokens': 671571.0, 'mean_token_accuracy': 0.8642715364694595, 'epoch': 4.53}
{'loss': 0.2485, 'grad_norm': 2.258570432662964, 'learning_rate': 2.3344689290232037e-06, 'entropy': 0.401023181155324, 'num_tokens': 672857.0, 'mean_token_accuracy': 0.9312469363212585, 'epoch': 4.54}
{'loss': 0.4062, 'grad_norm': 2.9382598400115967, 'learning_rate': 2.2761922426948355e-06, 'entropy': 0.45729222148656845, 'num_tokens': 674087.0, 'mean_token_accuracy': 0.8964986354112625, 'epoch': 4.54}
{'loss': 0.3859, 'grad_norm': 2.46163272857666, 'learning_rate': 2.218635232737781e-06, 'entropy': 0.5284511037170887, 'num_tokens': 675283.0, 'mean_token_accuracy': 0.9020341038703918, 'epoch': 4.55}
{'loss': 0.3047, 'grad_norm': 2.5449044704437256, 'learning_rate': 2.1617987671133657e-06, 'entropy': 0.4014103338122368, 'num_tokens': 676456.0, 'mean_token_accuracy': 0.9170538783073425, 'epoch': 4.55}
{'loss': 0.4823, 'grad_norm': 3.310638666152954, 'learning_rate': 2.105683702917105e-06, 'entropy': 0.549021303653717, 'num_tokens': 677613.0, 'mean_token_accuracy': 0.8532290309667587, 'epoch': 4.56}
{'loss': 0.4276, 'grad_norm': 3.000593662261963, 'learning_rate': 2.050290886365752e-06, 'entropy': 0.5108906030654907, 'num_tokens': 678733.0, 'mean_token_accuracy': 0.8801403194665909, 'epoch': 4.57}
{'loss': 0.2547, 'grad_norm': 2.276695489883423, 'learning_rate': 1.9956211527845546e-06, 'entropy': 0.3678657878190279, 'num_tokens': 679776.0, 'mean_token_accuracy': 0.9404515624046326, 'epoch': 4.57}
{'loss': 0.3648, 'grad_norm': 2.6167259216308594, 'learning_rate': 1.941675326594633e-06, 'entropy': 0.48397909849882126, 'num_tokens': 680777.0, 'mean_token_accuracy': 0.9092579782009125, 'epoch': 4.58}
{'loss': 0.5417, 'grad_norm': 3.222170829772949, 'learning_rate': 1.8884542213005674e-06, 'entropy': 0.6461728513240814, 'num_tokens': 681738.0, 'mean_token_accuracy': 0.8490208983421326, 'epoch': 4.58}
{'loss': 0.3995, 'grad_norm': 3.3738090991973877, 'learning_rate': 1.8359586394781525e-06, 'entropy': 0.5283539369702339, 'num_tokens': 682681.0, 'mean_token_accuracy': 0.9007991999387741, 'epoch': 4.59}
{'loss': 0.316, 'grad_norm': 2.7308356761932373, 'learning_rate': 1.7841893727622238e-06, 'entropy': 0.44951028376817703, 'num_tokens': 683587.0, 'mean_token_accuracy': 0.9145930409431458, 'epoch': 4.6}
{'loss': 0.3841, 'grad_norm': 2.9756884574890137, 'learning_rate': 1.733147201834806e-06, 'entropy': 0.567306712269783, 'num_tokens': 684477.0, 'mean_token_accuracy': 0.9062495231628418, 'epoch': 4.6}
{'loss': 0.2765, 'grad_norm': 2.43131947517395, 'learning_rate': 1.6828328964132666e-06, 'entropy': 0.4071207791566849, 'num_tokens': 685347.0, 'mean_token_accuracy': 0.928652286529541, 'epoch': 4.61}
{'loss': 0.2495, 'grad_norm': 3.1824450492858887, 'learning_rate': 1.63324721523877e-06, 'entropy': 0.38751646131277084, 'num_tokens': 686198.0, 'mean_token_accuracy': 0.9338281452655792, 'epoch': 4.61}
{'loss': 0.3336, 'grad_norm': 2.945868968963623, 'learning_rate': 1.5843909060647867e-06, 'entropy': 0.4280775561928749, 'num_tokens': 687018.0, 'mean_token_accuracy': 0.9130968153476715, 'epoch': 4.62}
{'loss': 0.3314, 'grad_norm': 2.72786021232605, 'learning_rate': 1.536264705645829e-06, 'entropy': 0.4815397411584854, 'num_tokens': 687828.0, 'mean_token_accuracy': 0.9008053839206696, 'epoch': 4.63}
{'loss': 0.3823, 'grad_norm': 3.03910756111145, 'learning_rate': 1.4888693397263787e-06, 'entropy': 0.5058518499135971, 'num_tokens': 688622.0, 'mean_token_accuracy': 0.9036234319210052, 'epoch': 4.63}
{'loss': 0.2762, 'grad_norm': 2.6585302352905273, 'learning_rate': 1.442205523029877e-06, 'entropy': 0.36189382523298264, 'num_tokens': 689393.0, 'mean_token_accuracy': 0.9231476038694382, 'epoch': 4.64}
{'loss': 0.2903, 'grad_norm': 2.615379810333252, 'learning_rate': 1.3962739592480178e-06, 'entropy': 0.4322488158941269, 'num_tokens': 690123.0, 'mean_token_accuracy': 0.9298895001411438, 'epoch': 4.64}
{'loss': 0.2616, 'grad_norm': 2.6272389888763428, 'learning_rate': 1.3510753410300725e-06, 'entropy': 0.4069097936153412, 'num_tokens': 690824.0, 'mean_token_accuracy': 0.9384091198444366, 'epoch': 4.65}
{'loss': 0.2007, 'grad_norm': 2.5129237174987793, 'learning_rate': 1.3066103499724814e-06, 'entropy': 0.3286367692053318, 'num_tokens': 691507.0, 'mean_token_accuracy': 0.9440306127071381, 'epoch': 4.66}
{'loss': 0.3634, 'grad_norm': 2.7804152965545654, 'learning_rate': 1.2628796566085787e-06, 'entropy': 0.5403663516044617, 'num_tokens': 692186.0, 'mean_token_accuracy': 0.9140955060720444, 'epoch': 4.66}
{'loss': 0.1869, 'grad_norm': 2.5631062984466553, 'learning_rate': 1.2198839203984401e-06, 'entropy': 0.329427033662796, 'num_tokens': 692846.0, 'mean_token_accuracy': 0.9544863551855087, 'epoch': 4.67}
{'loss': 0.2744, 'grad_norm': 2.6016080379486084, 'learning_rate': 1.1776237897190011e-06, 'entropy': 0.4592963084578514, 'num_tokens': 693480.0, 'mean_token_accuracy': 0.9317125231027603, 'epoch': 4.67}
{'loss': 0.1512, 'grad_norm': 2.165189027786255, 'learning_rate': 1.1360999018542151e-06, 'entropy': 0.2996266633272171, 'num_tokens': 694091.0, 'mean_token_accuracy': 0.9719953387975693, 'epoch': 4.68}
{'loss': 0.3154, 'grad_norm': 3.016089916229248, 'learning_rate': 1.095312882985494e-06, 'entropy': 0.4326515421271324, 'num_tokens': 694688.0, 'mean_token_accuracy': 0.9277551621198654, 'epoch': 4.69}
{'loss': 0.2036, 'grad_norm': 2.6612746715545654, 'learning_rate': 1.0552633481822395e-06, 'entropy': 0.31558115780353546, 'num_tokens': 695262.0, 'mean_token_accuracy': 0.9439253062009811, 'epoch': 4.69}
{'loss': 0.182, 'grad_norm': 2.704057455062866, 'learning_rate': 1.015951901392559e-06, 'entropy': 0.36077797412872314, 'num_tokens': 695824.0, 'mean_token_accuracy': 0.9569501429796219, 'epoch': 4.7}
{'loss': 0.1643, 'grad_norm': 2.6631529331207275, 'learning_rate': 9.773791354341865e-07, 'entropy': 0.3110715448856354, 'num_tokens': 696373.0, 'mean_token_accuracy': 0.9614059776067734, 'epoch': 4.7}
{'loss': 0.2133, 'grad_norm': 2.626802921295166, 'learning_rate': 9.395456319855278e-07, 'entropy': 0.38085517287254333, 'num_tokens': 696887.0, 'mean_token_accuracy': 0.9508925974369049, 'epoch': 4.71}
{'loss': 0.1949, 'grad_norm': 2.7071311473846436, 'learning_rate': 9.024519615768834e-07, 'entropy': 0.37114980816841125, 'num_tokens': 697359.0, 'mean_token_accuracy': 0.9550860524177551, 'epoch': 4.72}
{'loss': 0.1389, 'grad_norm': 2.250070571899414, 'learning_rate': 8.660986835818508e-07, 'entropy': 0.2619774118065834, 'num_tokens': 697815.0, 'mean_token_accuracy': 0.971118301153183, 'epoch': 4.72}
{'loss': 0.106, 'grad_norm': 1.4721461534500122, 'learning_rate': 8.304863462088974e-07, 'entropy': 0.18766602873802185, 'num_tokens': 698234.0, 'mean_token_accuracy': 0.982932522892952, 'epoch': 4.73}
{'loss': 0.1041, 'grad_norm': 1.8790711164474487, 'learning_rate': 7.95615486493062e-07, 'entropy': 0.20776094496250153, 'num_tokens': 698585.0, 'mean_token_accuracy': 0.9770849198102951, 'epoch': 4.73}
{'loss': 0.132, 'grad_norm': 2.0353729724884033, 'learning_rate': 7.614866302879165e-07, 'entropy': 0.23638597503304482, 'num_tokens': 698900.0, 'mean_token_accuracy': 0.9804616868495941, 'epoch': 4.74}
{'loss': 0.6903, 'grad_norm': 2.3842527866363525, 'learning_rate': 7.281002922575609e-07, 'entropy': 0.7239382266998291, 'num_tokens': 701446.0, 'mean_token_accuracy': 0.7989648431539536, 'epoch': 4.75}
{'loss': 0.5765, 'grad_norm': 2.2653558254241943, 'learning_rate': 6.9545697586893e-07, 'entropy': 0.5707628205418587, 'num_tokens': 703532.0, 'mean_token_accuracy': 0.8430771380662918, 'epoch': 4.75}
{'loss': 0.4922, 'grad_norm': 2.4795327186584473, 'learning_rate': 6.63557173384155e-07, 'entropy': 0.5738471522927284, 'num_tokens': 705238.0, 'mean_token_accuracy': 0.8561344891786575, 'epoch': 4.76}
{'loss': 0.4783, 'grad_norm': 2.0865509510040283, 'learning_rate': 6.324013658531525e-07, 'entropy': 0.5374535024166107, 'num_tokens': 706730.0, 'mean_token_accuracy': 0.8667332828044891, 'epoch': 4.76}
{'loss': 0.3537, 'grad_norm': 2.5878801345825195, 'learning_rate': 6.019900231063968e-07, 'entropy': 0.4380733519792557, 'num_tokens': 708085.0, 'mean_token_accuracy': 0.9037602245807648, 'epoch': 4.77}
{'loss': 0.4288, 'grad_norm': 2.5877394676208496, 'learning_rate': 5.723236037477764e-07, 'entropy': 0.5385594293475151, 'num_tokens': 709346.0, 'mean_token_accuracy': 0.887147068977356, 'epoch': 4.78}
{'loss': 0.4843, 'grad_norm': 3.09977126121521, 'learning_rate': 5.43402555147754e-07, 'entropy': 0.5858924835920334, 'num_tokens': 710548.0, 'mean_token_accuracy': 0.8874077498912811, 'epoch': 4.78}
{'loss': 0.47, 'grad_norm': 2.8544812202453613, 'learning_rate': 5.152273134365504e-07, 'entropy': 0.5673889145255089, 'num_tokens': 711716.0, 'mean_token_accuracy': 0.8644917011260986, 'epoch': 4.79}
{'loss': 0.367, 'grad_norm': 2.865952253341675, 'learning_rate': 4.877983034976053e-07, 'entropy': 0.48354023694992065, 'num_tokens': 712850.0, 'mean_token_accuracy': 0.9000144600868225, 'epoch': 4.79}
{'loss': 0.3519, 'grad_norm': 2.718912124633789, 'learning_rate': 4.611159389611763e-07, 'entropy': 0.4566574692726135, 'num_tokens': 713943.0, 'mean_token_accuracy': 0.8979915380477905, 'epoch': 4.8}
{'loss': 0.5296, 'grad_norm': 2.8676934242248535, 'learning_rate': 4.35180622198067e-07, 'entropy': 0.5968755930662155, 'num_tokens': 715012.0, 'mean_token_accuracy': 0.8619899302721024, 'epoch': 4.81}
{'loss': 0.3018, 'grad_norm': 2.870126247406006, 'learning_rate': 4.09992744313592e-07, 'entropy': 0.4351360946893692, 'num_tokens': 716064.0, 'mean_token_accuracy': 0.9209247529506683, 'epoch': 4.81}
{'loss': 0.3485, 'grad_norm': 2.768554210662842, 'learning_rate': 3.855526851416547e-07, 'entropy': 0.493310309946537, 'num_tokens': 717076.0, 'mean_token_accuracy': 0.9048748463392258, 'epoch': 4.82}
{'loss': 0.3748, 'grad_norm': 3.1818599700927734, 'learning_rate': 3.618608132390511e-07, 'entropy': 0.46598079800605774, 'num_tokens': 718057.0, 'mean_token_accuracy': 0.9098857045173645, 'epoch': 4.82}
{'loss': 0.3332, 'grad_norm': 3.0055959224700928, 'learning_rate': 3.389174858798694e-07, 'entropy': 0.40527207404375076, 'num_tokens': 719008.0, 'mean_token_accuracy': 0.9114496111869812, 'epoch': 4.83}
{'loss': 0.262, 'grad_norm': 2.663491725921631, 'learning_rate': 3.1672304905013825e-07, 'entropy': 0.3588431812822819, 'num_tokens': 719931.0, 'mean_token_accuracy': 0.9296605885028839, 'epoch': 4.84}
{'loss': 0.4121, 'grad_norm': 2.823302745819092, 'learning_rate': 2.95277837442598e-07, 'entropy': 0.5139402374625206, 'num_tokens': 720804.0, 'mean_token_accuracy': 0.8961009085178375, 'epoch': 4.84}
{'loss': 0.315, 'grad_norm': 2.8396873474121094, 'learning_rate': 2.7458217445163745e-07, 'entropy': 0.40744750015437603, 'num_tokens': 721640.0, 'mean_token_accuracy': 0.9132610261440277, 'epoch': 4.85}
{'loss': 0.2865, 'grad_norm': 2.8405137062072754, 'learning_rate': 2.5463637216845413e-07, 'entropy': 0.33709732070565224, 'num_tokens': 722446.0, 'mean_token_accuracy': 0.9300608932971954, 'epoch': 4.85}
{'loss': 0.2438, 'grad_norm': 2.8586313724517822, 'learning_rate': 2.3544073137629073e-07, 'entropy': 0.38318033516407013, 'num_tokens': 723235.0, 'mean_token_accuracy': 0.9388982653617859, 'epoch': 4.86}
{'loss': 0.3876, 'grad_norm': 3.0422720909118652, 'learning_rate': 2.1699554154597234e-07, 'entropy': 0.5573834627866745, 'num_tokens': 724000.0, 'mean_token_accuracy': 0.8999600559473038, 'epoch': 4.87}
{'loss': 0.2845, 'grad_norm': 2.7603042125701904, 'learning_rate': 1.9930108083147104e-07, 'entropy': 0.44551148265600204, 'num_tokens': 724739.0, 'mean_token_accuracy': 0.9307835549116135, 'epoch': 4.87}
{'loss': 0.5094, 'grad_norm': 3.741255521774292, 'learning_rate': 1.8235761606576473e-07, 'entropy': 0.6034619808197021, 'num_tokens': 725447.0, 'mean_token_accuracy': 0.8778362572193146, 'epoch': 4.88}
{'loss': 0.2765, 'grad_norm': 5.143671989440918, 'learning_rate': 1.6616540275677384e-07, 'entropy': 0.3361550457775593, 'num_tokens': 726134.0, 'mean_token_accuracy': 0.9357379525899887, 'epoch': 4.88}
{'loss': 0.1643, 'grad_norm': 1.961991786956787, 'learning_rate': 1.5072468508353089e-07, 'entropy': 0.2847267873585224, 'num_tokens': 726802.0, 'mean_token_accuracy': 0.9563950598239899, 'epoch': 4.89}
{'loss': 0.2626, 'grad_norm': 3.282785654067993, 'learning_rate': 1.3603569589250575e-07, 'entropy': 0.4019249379634857, 'num_tokens': 727439.0, 'mean_token_accuracy': 0.9338845163583755, 'epoch': 4.9}
{'loss': 0.2528, 'grad_norm': 3.131863832473755, 'learning_rate': 1.220986566940696e-07, 'entropy': 0.35548578947782516, 'num_tokens': 728056.0, 'mean_token_accuracy': 0.9266498535871506, 'epoch': 4.9}
{'loss': 0.2559, 'grad_norm': 2.8120226860046387, 'learning_rate': 1.0891377765915866e-07, 'entropy': 0.36597028002142906, 'num_tokens': 728661.0, 'mean_token_accuracy': 0.9317659884691238, 'epoch': 4.91}
{'loss': 0.1934, 'grad_norm': 2.5616917610168457, 'learning_rate': 9.648125761612669e-08, 'entropy': 0.39282023161649704, 'num_tokens': 729257.0, 'mean_token_accuracy': 0.9527461975812912, 'epoch': 4.91}
{'loss': 0.1849, 'grad_norm': 2.997828722000122, 'learning_rate': 8.48012840477308e-08, 'entropy': 0.26234667003154755, 'num_tokens': 729842.0, 'mean_token_accuracy': 0.958630695939064, 'epoch': 4.92}
{'loss': 0.2167, 'grad_norm': 2.7547430992126465, 'learning_rate': 7.387403308829477e-08, 'entropy': 0.3714747503399849, 'num_tokens': 730406.0, 'mean_token_accuracy': 0.9428428262472153, 'epoch': 4.93}
{'loss': 0.1808, 'grad_norm': 2.720386028289795, 'learning_rate': 6.369966952108342e-08, 'entropy': 0.3215309977531433, 'num_tokens': 730955.0, 'mean_token_accuracy': 0.9559234976768494, 'epoch': 4.93}
{'loss': 0.1335, 'grad_norm': 2.1209568977355957, 'learning_rate': 5.4278346775782365e-08, 'entropy': 0.23657451942563057, 'num_tokens': 731467.0, 'mean_token_accuracy': 0.9685842841863632, 'epoch': 4.94}
{'loss': 0.2085, 'grad_norm': 3.2465362548828125, 'learning_rate': 4.5610206926199885e-08, 'entropy': 0.3831360191106796, 'num_tokens': 731959.0, 'mean_token_accuracy': 0.9404874444007874, 'epoch': 4.94}
{'loss': 0.1311, 'grad_norm': 2.14284348487854, 'learning_rate': 3.7695380688135274e-08, 'entropy': 0.25714248046278954, 'num_tokens': 732433.0, 'mean_token_accuracy': 0.970210999250412, 'epoch': 4.95}
{'loss': 0.1596, 'grad_norm': 2.2258963584899902, 'learning_rate': 3.053398741739155e-08, 'entropy': 0.28964000940322876, 'num_tokens': 732892.0, 'mean_token_accuracy': 0.9669675379991531, 'epoch': 4.96}
{'loss': 0.1254, 'grad_norm': 1.7104756832122803, 'learning_rate': 2.412613510798245e-08, 'entropy': 0.2218172550201416, 'num_tokens': 733329.0, 'mean_token_accuracy': 0.972263514995575, 'epoch': 4.96}
{'loss': 0.1688, 'grad_norm': 3.4848215579986572, 'learning_rate': 1.8471920390505938e-08, 'entropy': 0.2848406322300434, 'num_tokens': 733750.0, 'mean_token_accuracy': 0.9542745053768158, 'epoch': 4.97}
{'loss': 0.0948, 'grad_norm': 1.9775604009628296, 'learning_rate': 1.3571428530667617e-08, 'entropy': 0.16941187903285027, 'num_tokens': 734134.0, 'mean_token_accuracy': 0.9894212335348129, 'epoch': 4.97}
{'loss': 0.1322, 'grad_norm': 2.0865437984466553, 'learning_rate': 9.424733428037291e-09, 'entropy': 0.24121407605707645, 'num_tokens': 734471.0, 'mean_token_accuracy': 0.960950955748558, 'epoch': 4.98}
{'loss': 0.1603, 'grad_norm': 1.8154258728027344, 'learning_rate': 6.031897614883209e-09, 'entropy': 0.21043771877884865, 'num_tokens': 734772.0, 'mean_token_accuracy': 0.9729006886482239, 'epoch': 4.99}
{'loss': 0.2366, 'grad_norm': 2.8629870414733887, 'learning_rate': 3.392972255272797e-09, 'entropy': 0.32777468860149384, 'num_tokens': 735580.0, 'mean_token_accuracy': 0.9322230815887451, 'epoch': 4.99}
{'loss': 0.1515, 'grad_norm': 2.386309862136841, 'learning_rate': 1.5079971442732988e-09, 'entropy': 0.29878927394747734, 'num_tokens': 736100.0, 'mean_token_accuracy': 0.9630796313285828, 'epoch': 5.0}
{'loss': 0.0992, 'grad_norm': 3.117526054382324, 'learning_rate': 3.770007073633508e-10, 'entropy': 0.1386963427066803, 'num_tokens': 736175.0, 'mean_token_accuracy': 0.9729729890823364, 'epoch': 5.0}
{'train_runtime': 4340.5655, 'train_samples_per_second': 0.766, 'train_steps_per_second': 0.192, 'train_loss': 0.929355051756619, 'epoch': 5.0}
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 835/835 [1:12:20<00:00,  5.20s/it]
Saving adapter to finetuned_adapter...
Done! Merge this adapter or load it in Kobold.

B:\8B\!models--SicariusSicariiStuff--Llama-3.1-Nemotron-8B-UltraLong-1M-Instruct_Abliterated>