codebert-base-finetuned-code-ner-15e

This model is a fine-tuned version of microsoft/codebert-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3831
• Precision: 0.6363
• Recall: 0.6494
• F1: 0.6428
• Accuracy: 0.9197

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
No log	1.0	191	0.4566	0.5021	0.4220	0.4585	0.8827
No log	2.0	382	0.3756	0.5699	0.5764	0.5731	0.9043
0.5133	3.0	573	0.3605	0.6001	0.5767	0.5882	0.9093
0.5133	4.0	764	0.3500	0.6130	0.6130	0.6130	0.9153
0.5133	5.0	955	0.3501	0.6337	0.6172	0.6254	0.9178
0.2203	6.0	1146	0.3645	0.6250	0.6352	0.6300	0.9163
0.2203	7.0	1337	0.3488	0.6263	0.6422	0.6341	0.9189
0.1457	8.0	1528	0.3575	0.6372	0.6397	0.6384	0.9194
0.1457	9.0	1719	0.3662	0.6406	0.6343	0.6375	0.9189
0.1457	10.0	1910	0.3613	0.6374	0.6473	0.6423	0.9201
0.107	11.0	2101	0.3716	0.6329	0.6544	0.6435	0.9197
0.107	12.0	2292	0.3754	0.6328	0.6487	0.6406	0.9193
0.107	13.0	2483	0.3826	0.6395	0.6490	0.6443	0.9204
0.0863	14.0	2674	0.3821	0.6368	0.6535	0.6451	0.9200
0.0863	15.0	2865	0.3831	0.6363	0.6494	0.6428	0.9197

Evaluation results

	Algorithm	Application	Class	Code_Block	Data_Structure	Data_Type	Device	Error_Name	File_Name	File_Type	Function	HTML_XML_Tag	Keyboard_IP	Language	Library	Operating_System	Output_Block	User_Interface_Element	User_Name	Value	Variable	Version	Website	overall_precision	overall_recall	overall_f1	overall_accuracy
precision	0	0.619835	0.680851	0.455629	0.813187	0.592593	0.395062	0.181818	0.800505	0.775956	0.757664	0.585366	0.333333	0.689769	0.61807	0.769231	0.0212766	0.542214	0.4375	0.370236	0.560479	0.883721	0.382353	0.626308	0.642171	0.63414	0.918927
recall	0	0.677711	0.696864	0.494253	0.840909	0.8	0.533333	0.333333	0.794486	0.628319	0.631387	0.470588	0.0169492	0.81323	0.546279	0.843373	0.04	0.653846	0.518519	0.52987	0.54482	0.914089	0.270833	0.626308	0.642171	0.63414	0.918927
f1	0	0.647482	0.688765	0.474156	0.826816	0.680851	0.453901	0.235294	0.797484	0.694377	0.688786	0.521739	0.0322581	0.746429	0.579961	0.804598	0.0277778	0.592821	0.474576	0.435897	0.552538	0.898649	0.317073	0.626308	0.642171	0.63414	0.918927
number	31	664	1148	696	264	120	60	30	798	226	822	102	59	257	551	83	25	442	54	385	859	291	48	0.626308	0.642171	0.63414	0.918927

Framework versions

• Transformers 4.23.1
• Pytorch 1.12.1+cu113
• Datasets 2.6.1
• Tokenizers 0.13.1