qwen3_4_20250818_1041

This model is a fine-tuned version of Qwen/Qwen3-4B on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3702
• Map@3: 0.9280

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 64
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Map@3
17.122	0.0598	20	1.3226	0.7370
9.722	0.1196	40	0.8734	0.8255
6.7477	0.1794	60	0.8755	0.8327
6.0617	0.2392	80	0.7237	0.8536
5.5633	0.2990	100	0.6814	0.8678
5.4724	0.3587	120	0.6710	0.8683
4.9076	0.4185	140	0.6206	0.8762
4.557	0.4783	160	0.6088	0.8871
4.5098	0.5381	180	0.5562	0.8925
4.417	0.5979	200	0.5503	0.8943
4.0431	0.6577	220	0.4958	0.9056
4.2156	0.7175	240	0.4706	0.9100
4.305	0.7773	260	0.4575	0.9137
3.7346	0.8371	280	0.5350	0.9048
3.9774	0.8969	300	0.4450	0.9153
3.7675	0.9567	320	0.4582	0.9137
3.3745	1.0149	340	0.4185	0.9185
2.7309	1.0747	360	0.4246	0.9187
2.3964	1.1345	380	0.4226	0.9209
3.4858	1.1943	400	0.4100	0.9231
2.4694	1.2541	420	0.4273	0.9231
2.6601	1.3139	440	0.4052	0.9199
2.7272	1.3737	460	0.3917	0.9289
2.4134	1.4335	480	0.4046	0.9232
2.5493	1.4933	500	0.3673	0.9276
2.5291	1.5531	520	0.3725	0.9273
2.3224	1.6129	540	0.3702	0.9280

Framework versions

• PEFT 0.17.0
• Transformers 4.55.2
• Pytorch 2.6.0+cu124
• Datasets 4.0.0
• Tokenizers 0.21.4