Update README.md

README.md

@@ -1,3 +1,62 @@
----
-license: llama3
----
+---
+model-index:
+- name: Junrulu/Llama-3-8B-Instruct-Iterative-SamPO
+  results: []
+datasets:
+- HuggingFaceH4/ultrafeedback_binarized
+language:
+- en
+base_model: meta-llama/Meta-Llama-3-8B-Instruct
+license: llama3
+---
+
+# Model Card for Llama-3-8B-Instruct-Iterative-SamPO
+
+This repository provides a fine-tuned version of Llama-3-8B-Instruct, using our proposed [SamPO](https://github.com/LuJunru/SamPO) algorithm. We obey all licenses mentioned in llama3's work.
+
+## Performance
+
+| Model | GSM8K | IFEval | PiQA | MMLU | TruthfulQA | AlpacaEval2 | LC AlpacaEval2 | Length in Tokens |
+| ----- | ------| ------ | ---- | ---- | ---------- | ----------- | -------------- | ---------------- |
+| **Llama3-8B-Instruct** | 75.06 | 49.40 | 80.69 | 63.85 | 36.47 | 22.57 | 22.92 | 421 |
+| **Llama3-8B-Instruct-DPO** | 75.59 | 51.80 | **81.94** | 64.06 | 40.39 | 23.34 | 23.20 | 422 |
+| **Llama3-8B-Instruct-Iterative-DPO** | 74.91 | 52.52 | 81.66 | 64.02 | 39.90 | 23.92 | 25.50 | 403 |
+| **Llama3-8B-Instruct-Iterative-SamPO** | **77.81** | **60.55** | 81.18 | **64.12** | **44.07** | **30.68** | **35.14** | 377 |
+
+## Evaluation Details
+Five conditional benchmarks, using [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness):
+- GSM8K: 8-shot, report strict match
+- IFEval: 3-shot, report instruction-level strict accuracy
+- PiQA: 3-shot, report accuracy
+- MMLU: 0-shot, report normalized accuracy
+- TruthfulQA: 3-shot, report accuracy of single-true mc1 setting
+
+One open-ended benchmark, using official [alpaca_eval](https://github.com/tatsu-lab/alpaca_eval/):
+- AlpacaEval2: win rate (%) judged by GPT-4-turbo between the model's outputs vs. the GPT-4-turbo's response
+- LC AlpacaEval2: length-debiased win rate (%) of AlpacaEval2
+- Length in Tokens: the average output length of AlpacaEval2, calculated in tokens with Llama3's tokenizer
+
+## Input Format
+
+The model is trained to use the following format:
+```
+<|start_header_id|>user<|end_header_id|>
+
+{PROMPT}<|eot_id|>
+<|start_header_id|>assistant<|end_header_id|>
+
+{Response}
+```
+
+## Training hyperparameters
+
+The following hyperparameters were used during DPO/SamPO training:
+- DPO beta: 0.1 
+- learning_rate: 4e-7 * sqrt(Num of Nodes)
+- total_train_batch_size: 128 * Num of Nodes
+- optimizer: AdamW with beta1 0.9, beta2 0.999 and epsilon 1e-8
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_ratio: 0.1
+- Weight Decay: 0.0
+- num_epochs: 3.0
+- Specifically add above input format over training samples