qwen3_4_20250810_1540

This model is a fine-tuned version of Qwen/Qwen3-4B on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4485
• Map@3: 0.9108

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 64
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Map@3
18.0014	0.0598	20	1.5174	0.6987
10.6701	0.1196	40	0.9999	0.7962
7.6249	0.1794	60	0.8878	0.8231
6.7046	0.2392	80	0.8309	0.8358
5.9628	0.2990	100	0.6635	0.8675
5.3938	0.3587	120	0.7054	0.8613
4.7895	0.4185	140	0.5530	0.8877
4.8493	0.4783	160	0.5481	0.8944
4.3363	0.5381	180	0.5117	0.8993
4.2649	0.5979	200	0.5654	0.8960
4.0245	0.6577	220	0.4839	0.9053
4.4226	0.7175	240	0.4850	0.9103
3.9435	0.7773	260	0.4436	0.9176
3.6812	0.8371	280	0.4542	0.9135
3.8898	0.8969	300	0.4466	0.9129
3.9869	0.9567	320	0.4188	0.9211
3.371	1.0149	340	0.4492	0.9113
2.6116	1.0747	360	0.4368	0.9203
2.5455	1.1345	380	0.4300	0.9148
3.2972	1.1943	400	0.4485	0.9108

Framework versions

• PEFT 0.17.0
• Transformers 4.55.0
• Pytorch 2.6.0+cu124
• Datasets 4.0.0
• Tokenizers 0.21.4