Improve language tag

Hi! As the model is multilingual, this is a PR to add other languages than English to the language tag to improve the referencing. Note that 29 languages are announced in the README, but only 13 are explicitly listed. I was therefore only able to add these 13 languages.

README.md

@@ -1,128 +1,139 @@
----
-library_name: transformers
-license: mit
-datasets:
-- Geraldine/Ead-Instruct-4k-Distilled
-language:
-- fr
-- en
-base_model:
-- Qwen/Qwen2.5-0.5B-Instruct
----
-
-# Gemini-Distill-Qwen2.5-0.5B-ead : Qwen2.5-0.5B-Instruct Fine-Tuned on EAD/XML (Distilled from Gemini-2.0-Flash-Thinking-Exp)
-
-## Model Description
-This model is a fine-tuned version of **Qwen2.5-0.5B-Instruct**, trained via knowledge distillation from **Gemini-2.0-Flash-Thinking-Exp**. The goal of this fine-tuning process is to teach the model to reason through and generate **Encoded Archival Description (EAD/XML)** outputs.
-
-It follows a structured reasoning approach:
-1. **First**, the model provides detailed reasoning.
-2. **Then**, it outputs the final **EAD/XML** response.
-
-This structure ensures that the model justifies its output before producing the archival XML format, improving interpretability and accuracy.
-
----
-## Training Details
-
-### **Dataset**
-- Dataset: [Geraldine/Ead-Instruct-4k-Distilled](https://huggingface.co/datasets/Geraldine/Ead-Instruct-4k-Distilled)
-- **Columns:**
-  - `tag`: EAD/XML element
-  - `prompt`: User query
-  - `reasoning`: Gemini-generated reasoning traces
-  - `final_output`: EAD/XML archival response
-  - `completion`: Concatenation of `reasoning` and `final_output`
-
-### **Training Process**
-- **Hardware:** : NVIDIA A100-SXM4-80GB
-- **Distillation Source:** Gemini-2.0-Flash-Thinking-Exp
-- **Model trained on:** User prompt → Gemini reasoning traces → Final EAD/XML response
-- **Tokenization Strategy:**
-  - **Assistant (reasoning):** The start of the reasoning section
-  - **Assistant (final answer):** The start of the XML output
-  - Labels are masked (`-100`) before the reasoning phase to optimize the learning process
-- **Training Hyperparameters:**
-  - **Batch Size:** 4 (per device) with gradient accumulation (steps=2)
-  - **Max Sequence Length:** 4096 tokens
-  - **Precision:** bf16
-  - **Epochs:** 5
-  - **Gradient Checkpointing:** Enabled (reduces memory usage)
-  - **Dataloader Efficiency:** dataloader_pin_memory=True, dataloader_num_workers=4
-  - **Warmup Steps:** 100
-  - **Checkpointing:** Model saved at every epoch, with a maximum of 2 saved checkpoints (save_total_limit=2)
-  - **Evaluation Strategy:** Evaluates after each epoch (eval_strategy="epoch")
-  - **Logging:** Logs stored in ./logs
-  - **Other:** dataloader_drop_last=False to preserve all batches
-
-This setup ensures an optimal balance between performance and memory efficiency, leveraging gradient accumulation and checkpointing for stable training on long sequences
-
-### **Training notebook**
-
-[https://www.kaggle.com/code/geraldinegeoffroy/ead-distilled-qwen2-5-0-5b-instruct](https://www.kaggle.com/code/geraldinegeoffroy/ead-distilled-qwen2-5-0-5b-instruct)
-
----
-## Model Usage
-
-### **Load Model**
-To use the model with the 🤗 Transformers library:
-```python
-from transformers import AutoModelForCausalLM, AutoTokenizer
-
-model_name = "Geraldine/Gemini-Distilled-Qwen2.5-0.5B-ead"
-tokenizer = AutoTokenizer.from_pretrained(model_name)
-model = AutoModelForCausalLM.from_pretrained(
-    model_name,
-    torch_dtype="auto",
-    device_map="auto", 
-)
-```
-
-### **Inference Example**
-```python
-tokenizer = AutoTokenizer.from_pretrained(model_name)
-
-prompt = "Give me an example of <controlaccess> content."
-messages = [
-    {"role": "system", "content": "You are an archivist expert in EAD/XML format for archival records metadata."},
-    {"role": "user", "content": prompt}
-]
-text = tokenizer.apply_chat_template(
-    messages,
-    tokenize=False,
-    add_generation_prompt=True
-)
-model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
-
-generated_ids = model.generate(
-    **model_inputs,
-    max_new_tokens=1024
-)
-generated_ids = [
-    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
-]
-
-response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
-print(response)
-```
-
----
-## Limitations & Future Improvements
-- **Training Data Size:** The dataset consists of **4,000 distilled samples**, which may limit generalization.
-- **Inference Speed:** Ensure that **Sliding Window Attention (SWA) is disabled**, as it may slow down inference.
-  - To disable: `model.config.sliding_window = None`
-- **Potential Future Steps:**
-  - Fine-tuning on larger datasets
-  - Exploring LoRA/QLoRA for efficient parameter tuning
-
----
-## Citation & Acknowledgments
-If you use this model in research or production, please cite:
-```
-@misc{your-citation,
-  author = {Géralidne Geoffroy},
-  title = {Qwen2.5-0.5B-Instruct Fine-Tuned on EAD/XML},
-  year = {2025},
-  publisher = {Hugging Face},
-  url = {https://huggingface.co/Geraldine/Gemini-Distill-Qwen2.5-0.5B-ead}
-}
+---
+library_name: transformers
+license: mit
+datasets:
+- Geraldine/Ead-Instruct-4k-Distilled
+language:
+- zho
+- eng
+- fra
+- spa
+- por
+- deu
+- ita
+- rus
+- jpn
+- kor
+- vie
+- tha
+- ara
+base_model:
+- Qwen/Qwen2.5-0.5B-Instruct
+---
+
+# Gemini-Distill-Qwen2.5-0.5B-ead : Qwen2.5-0.5B-Instruct Fine-Tuned on EAD/XML (Distilled from Gemini-2.0-Flash-Thinking-Exp)
+
+## Model Description
+This model is a fine-tuned version of **Qwen2.5-0.5B-Instruct**, trained via knowledge distillation from **Gemini-2.0-Flash-Thinking-Exp**. The goal of this fine-tuning process is to teach the model to reason through and generate **Encoded Archival Description (EAD/XML)** outputs.
+
+It follows a structured reasoning approach:
+1. **First**, the model provides detailed reasoning.
+2. **Then**, it outputs the final **EAD/XML** response.
+
+This structure ensures that the model justifies its output before producing the archival XML format, improving interpretability and accuracy.
+
+---
+## Training Details
+
+### **Dataset**
+- Dataset: [Geraldine/Ead-Instruct-4k-Distilled](https://huggingface.co/datasets/Geraldine/Ead-Instruct-4k-Distilled)
+- **Columns:**
+  - `tag`: EAD/XML element
+  - `prompt`: User query
+  - `reasoning`: Gemini-generated reasoning traces
+  - `final_output`: EAD/XML archival response
+  - `completion`: Concatenation of `reasoning` and `final_output`
+
+### **Training Process**
+- **Hardware:** : NVIDIA A100-SXM4-80GB
+- **Distillation Source:** Gemini-2.0-Flash-Thinking-Exp
+- **Model trained on:** User prompt → Gemini reasoning traces → Final EAD/XML response
+- **Tokenization Strategy:**
+  - **Assistant (reasoning):** The start of the reasoning section
+  - **Assistant (final answer):** The start of the XML output
+  - Labels are masked (`-100`) before the reasoning phase to optimize the learning process
+- **Training Hyperparameters:**
+  - **Batch Size:** 4 (per device) with gradient accumulation (steps=2)
+  - **Max Sequence Length:** 4096 tokens
+  - **Precision:** bf16
+  - **Epochs:** 5
+  - **Gradient Checkpointing:** Enabled (reduces memory usage)
+  - **Dataloader Efficiency:** dataloader_pin_memory=True, dataloader_num_workers=4
+  - **Warmup Steps:** 100
+  - **Checkpointing:** Model saved at every epoch, with a maximum of 2 saved checkpoints (save_total_limit=2)
+  - **Evaluation Strategy:** Evaluates after each epoch (eval_strategy="epoch")
+  - **Logging:** Logs stored in ./logs
+  - **Other:** dataloader_drop_last=False to preserve all batches
+
+This setup ensures an optimal balance between performance and memory efficiency, leveraging gradient accumulation and checkpointing for stable training on long sequences
+
+### **Training notebook**
+
+[https://www.kaggle.com/code/geraldinegeoffroy/ead-distilled-qwen2-5-0-5b-instruct](https://www.kaggle.com/code/geraldinegeoffroy/ead-distilled-qwen2-5-0-5b-instruct)
+
+---
+## Model Usage
+
+### **Load Model**
+To use the model with the 🤗 Transformers library:
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_name = "Geraldine/Gemini-Distilled-Qwen2.5-0.5B-ead"
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    torch_dtype="auto",
+    device_map="auto", 
+)
+```
+
+### **Inference Example**
+```python
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+
+prompt = "Give me an example of <controlaccess> content."
+messages = [
+    {"role": "system", "content": "You are an archivist expert in EAD/XML format for archival records metadata."},
+    {"role": "user", "content": prompt}
+]
+text = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True
+)
+model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
+
+generated_ids = model.generate(
+    **model_inputs,
+    max_new_tokens=1024
+)
+generated_ids = [
+    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
+]
+
+response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
+print(response)
+```
+
+---
+## Limitations & Future Improvements
+- **Training Data Size:** The dataset consists of **4,000 distilled samples**, which may limit generalization.
+- **Inference Speed:** Ensure that **Sliding Window Attention (SWA) is disabled**, as it may slow down inference.
+  - To disable: `model.config.sliding_window = None`
+- **Potential Future Steps:**
+  - Fine-tuning on larger datasets
+  - Exploring LoRA/QLoRA for efficient parameter tuning
+
+---
+## Citation & Acknowledgments
+If you use this model in research or production, please cite:
+```
+@misc{your-citation,
+  author = {Géralidne Geoffroy},
+  title = {Qwen2.5-0.5B-Instruct Fine-Tuned on EAD/XML},
+  year = {2025},
+  publisher = {Hugging Face},
+  url = {https://huggingface.co/Geraldine/Gemini-Distill-Qwen2.5-0.5B-ead}
+}
 ```