|
|
--- |
|
|
license: cc-by-nc-4.0 |
|
|
language: |
|
|
- tuk |
|
|
- eng |
|
|
library_name: transformers |
|
|
datasets: |
|
|
- XSkills/turkmen_english_s500 |
|
|
tags: |
|
|
- translation |
|
|
- nllb |
|
|
- lora |
|
|
- peft |
|
|
- turkmen |
|
|
model_name: nllb-200-turkmen-english-lora |
|
|
pipeline_tag: translation |
|
|
base_model: |
|
|
- facebook/nllb-200-distilled-600M |
|
|
--- |
|
|
|
|
|
# NLLB-200 (600 M) – LoRA fine-tuned for Turkmen ↔ English |
|
|
|
|
|
**Author** : Merdan Durdyyev |
|
|
**Base model** : [`facebook/nllb-200-distilled-600M`](https://huggingface.co/facebook/nllb-200-distilled-600M) |
|
|
**Tuning method** : Low-Rank Adaptation (LoRA) on only the `q_proj` & `v_proj` matrices (≈ 2.4 M trainable → 0.38 % of total params). |
|
|
|
|
|
> I built this checkpoint as the final project for my Deep-Learning class and as a small contribution to the Turkmen AI community, where open-source resources are scarce. |
|
|
|
|
|
--- |
|
|
|
|
|
## TL;DR & Quick results |
|
|
|
|
|
Try it on [Space demo](https://huggingface.co/spaces/XSkills/nllb-turkmen-english) Article with full technical journey is available [Medium](https://medium.com/@meinnps/fine-tuning-nllb-200-with-lora-on-a-650-sentence-turkmen-english-corpus-082f68bdec71). |
|
|
|
|
|
### Model Comparison (Fine-tuned vs Original) |
|
|
|
|
|
#### English to Turkmen |
|
|
|
|
|
| Metric | Fine-tuned | Original | Difference | |
|
|
|---------------------------|-----------:|---------:|-----------:| |
|
|
| **BLEU** | 8.24 | 8.12 | +0.12 | |
|
|
| **chrF** | 39.55 | 39.46 | +0.09 | |
|
|
| **TER (lower is better)** | 87.20 | 87.30 | -0.10 | |
|
|
|
|
|
#### Turkmen to English |
|
|
|
|
|
| Metric | Fine-tuned | Original | Difference | |
|
|
|---------------------------|-----------:|---------:|-----------:| |
|
|
| **BLEU** | 25.88 | 26.48 | -0.60 | |
|
|
| **chrF** | 52.71 | 52.91 | -0.20 | |
|
|
| **TER (lower is better)** | 67.70 | 69.70 | -2.00 | |
|
|
|
|
|
*Scores computed with sacre BLEU 2.5, chrF, TER on the official `test` split. |
|
|
A separate spreadsheet with **human adequacy/fluency ratings** is available in the article.* |
|
|
|
|
|
--- |
|
|
|
|
|
## Intended use & scope |
|
|
|
|
|
* **Good for**: research prototypes, student projects, quick experiments on Turkmen text. |
|
|
* **Not for**: commercial MT systems (license is **CC-BY-NC 4.0**), critical medical/legal translation, or production workloads without further validation. |
|
|
|
|
|
--- |
|
|
|
|
|
## How to use |
|
|
|
|
|
*(If you want to take a look to the LoRA adapter visit [nllb-200-turkmen-english-lora-adapter](https://huggingface.co/XSkills/nllb-200-turkmen-english-lora-adapter/tree/main))* |
|
|
|
|
|
Using piplene |
|
|
```python |
|
|
from transformers import pipeline |
|
|
|
|
|
# Create the translation pipeline |
|
|
pipe = pipeline("translation", model="XSkills/nllb-200-turkmen-english-lora") |
|
|
|
|
|
# Translate from English to Turkmen |
|
|
# You need to specify the source and target languages using their FLORES-200 codes |
|
|
text = "Hello, how are you today?" |
|
|
translated = pipe(text, src_lang="eng_Latn", tgt_lang="tuk_Latn") |
|
|
print(translated) |
|
|
``` |
|
|
|
|
|
Using Tokenizer |
|
|
```python |
|
|
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM |
|
|
|
|
|
model_id = "XSkills/nllb-200-turkmen-english-lora" |
|
|
tok = AutoTokenizer.from_pretrained(model_id) |
|
|
model = AutoModelForSeq2SeqLM.from_pretrained(model_id) |
|
|
|
|
|
def tr(text, src="tuk_Latn", tgt="eng_Latn"): |
|
|
tok.src_lang = src |
|
|
ids = tok(text, return_tensors="pt", truncation=True, max_length=128) |
|
|
out = model.generate( |
|
|
**ids, |
|
|
forced_bos_token_id=tok.convert_tokens_to_ids(tgt), |
|
|
max_length=128, |
|
|
num_beams=5 |
|
|
) |
|
|
return tok.decode(out[0], skip_special_tokens=True) |
|
|
|
|
|
print(tr("Men kitaby okaýaryn.")) |
|
|
``` |
|
|
|
|
|
## Training data |
|
|
- Dataset : [XSkills/turkmen_english_s500](https://huggingface.co/datasets/XSkills/turkmen_english_s500) 619 parallel sentences (495 train / 62 val / 62 test) of news & official communiqués. |
|
|
- Collecting even this small corpus proved challenging because publicly available Turkmen data are limited. |
|
|
|
|
|
## Training procedure |
|
|
|
|
|
| Item | Value | |
|
|
|------|-------| |
|
|
| GPU | 1 × NVIDIA A100 40 GB (Google Colab) | |
|
|
| Wall-time | ~ 3 minutes | |
|
|
| Optimiser | AdamW | |
|
|
| Learning rate | 1 × 10⁻⁵, cosine schedule, warm-up 10% | |
|
|
| Epochs | 5 | |
|
|
| Batch size | 4 (train) / 8 (eval) | |
|
|
| Weight-decay | 0.005 | |
|
|
| FP16 | Yes | |
|
|
| LoRA config | `r=16`, `alpha=32`, `dropout=0.05`, modules = `["q_proj","v_proj"]` | |
|
|
|
|
|
### LoRA Config |
|
|
|
|
|
```python |
|
|
lora_config = LoraConfig( |
|
|
r=16, |
|
|
lora_alpha=32, |
|
|
target_modules=["q_proj", "v_proj"], |
|
|
lora_dropout=0.05, |
|
|
bias="none", |
|
|
task_type=TaskType.SEQ_2_SEQ_LM, |
|
|
) |
|
|
``` |
|
|
|
|
|
### Training Configuration |
|
|
|
|
|
```python |
|
|
training_args = Seq2SeqTrainingArguments( |
|
|
output_dir=FINETUNED_DIR, |
|
|
per_device_train_batch_size=4, |
|
|
per_device_eval_batch_size=8, |
|
|
weight_decay=0.005, |
|
|
save_total_limit=3, |
|
|
learning_rate=1e-5, |
|
|
num_train_epochs=5, |
|
|
lr_scheduler_type="cosine", |
|
|
predict_with_generate=True, |
|
|
fp16=True if torch.cuda.is_available() else False, |
|
|
logging_dir="./logs", |
|
|
logging_steps=50, |
|
|
eval_steps=50, |
|
|
save_steps=100, |
|
|
eval_accumulation_steps=2, |
|
|
report_to="tensorboard", |
|
|
warmup_ratio=0.1, |
|
|
metric_for_best_model="eval_bleu", # Use BLEU for model selection |
|
|
greater_is_better=True, |
|
|
) |
|
|
``` |
|
|
## Evaluation |
|
|
|
|
|
Automatic metrics are given in TL;DR. |
|
|
A manual review on 50 random test sentences showed: |
|
|
- Adequacy: 36 / 50 translations judged “Good” or better. |
|
|
- Fluency: 38 / 50 sound natural to a native speaker. |
|
|
*(Full spreadsheet available — ask via contact below.)* |
|
|
|
|
|
|
|
|
## Limitations & bias |
|
|
- Only 500ish sentences → limited vocabulary & domain coverage. |
|
|
- May hallucinate proper nouns or numbers on longer inputs. |
|
|
- Gender/ politeness nuances not guaranteed. |
|
|
- CC-BY-NC licence forbids commercial use; respect Meta’s original terms. |
|
|
|
|
|
## How to Contribute |
|
|
|
|
|
We welcome contributions to improve Turkmen-English translation capabilities! Here's how you can help: |
|
|
|
|
|
### Data Contributions |
|
|
- **Read Dataset Contribution**: You can find the instructions for contributing to the dataset at [Dataset Readme](https://huggingface.co/datasets/XSkills/turkmen_english_s500/blob/main/README.md) |
|
|
|
|
|
### Code Contributions |
|
|
- **Hyperparameter experiments**: Try different LoRA configurations and document your results |
|
|
- **Evaluation**: Help with human evaluation of translation quality and fluency |
|
|
- **Bug fixes**: Report issues or submit fixes for the model implementation |
|
|
|
|
|
### Use Cases & Documentation |
|
|
- **Example applications**: Share how you're using the model for research or projects |
|
|
- **Domain-specific guides**: Create guides for using the model in specific domains |
|
|
- **Translation examples**: Share interesting or challenging translation examples |
|
|
|
|
|
### Getting Started |
|
|
1. Fork the repository |
|
|
2. Make your changes |
|
|
3. Submit a pull request with clear documentation of your contribution |
|
|
4. For data contributions, contact the maintainer directly |
|
|
|
|
|
All contributors will be acknowledged in the model documentation. Contact [meinnps@gmail.com](mailto:meinnps@gmail.com) with any questions or to discuss potential contributions. |
|
|
|
|
|
--- |
|
|
|
|
|
*Note: This model is licensed under CC-BY-NC-4.0, so all contributions must be compatible with non-commercial use only.* |
|
|
|
|
|
## Citation |
|
|
```bibtex |
|
|
@misc{durdyyev2025turkmenNLLBLoRA, |
|
|
title = {LoRA Fine‐tuning of NLLB‐200 for Turkmen–English Translation}, |
|
|
author = {Durdyyev, Merdan}, |
|
|
year = {2025}, |
|
|
url = {https://huggingface.co/XSkills/nllb-200-turkmen-english-lora} |
|
|
} |
|
|
``` |
|
|
|
|
|
## Contact |
|
|
If you have questions, suggestions or want to collaborate, please reach out through [e-mail](meinnps@gmail.com), [LinkedIn]( https://linkedin.com/in/merdandt) or [Telegram](https://t.me/merdandt). |
|
|
|
|
|
## Future Work |
|
|
- Try to tune on bigger dataset. |
|
|
- Try to tweak the hyperparameters |
|
|
- Use [sacreBLEU](https://github.com/mjpost/sacrebleu) metric |
|
|
|
