Update README.md (#1)

- Update README.md (1bd50f6ee4e0ddd675a53c3a000ecda37268d805)


Co-authored-by: Chen Chen <xiaoyan001@users.noreply.huggingface.co>

README.md

@@ -27,13 +27,19 @@ pipeline_tag: text-generation
 
 ## 📖 Introduction
 
-**UNO-Scorer** is a lightweight yet high-precision general scoring model developed as part of **UNO-Bench**. It is designed to efficiently automate the evaluation of Large Multimodal Models (LMMs) with minimal computational overhead.
+**UNO-Scorer** is a lightweight yet high-precision **LLM-based evaluation model** designed to efficiently automate the evaluation of Large Multimodal Models (LMMs) with minimal computational overhead.
+
+**Core Functionality:**
+- **Input**: Question + Reference Answer + Model Response
+- **Processing**: Analyzes correctness by **comparing each sub-question** against the reference answer
+- **Output**: Numerical score + Detailed evaluation reasoning for each sub-question
 
 Built upon the powerful **Qwen3-14B** backbone, UNO-Scorer is fine-tuned on 13K high-quality in-house data. It overcomes the limitations of traditional Overall Reward Models (ORMs) by supporting **6 distinct question types**, with particular excellence in **Multi-Step Open-Ended Questions (MO)**.
 
+
 ## 📊 Performance
 
-UNO-Scorer demonstrates superior performance in automated evaluation, particularly in handling complex **Multi-Step Open-Ended Questions**. We compared the accuracy of our scorer against other advanced evaluators:
+UNO-Scorer demonstrates superior performance in automated evaluation, particularly in handling complex **Multi-Step Open-Ended Questions**. We compared the accuracy of our scorer against other advanced evaluators on our test set:
 
 | Model | Accuracy |
 | :--- | :--- |
@@ -43,57 +49,180 @@ UNO-Scorer demonstrates superior performance in automated evaluation, particular
 
 Experiments show that UNO-Scorer surpasses even proprietary frontier models like GPT-4.1 in this specific evaluation domain with lower cost.
 
-
-
 ## 💻 Usage
 
-### 0. Quick Start
+### ⚡ Quick Start (HuggingFace Transformers)
+
+Get started with UNO-Scorer in just a few lines of code:
 
 ```bash
 pip install -U transformers
-python3 test_scorer_hf.py --model-name /path/to/your/model
+python3 test_scorer_hf.py --model-name /path/to/UNO-Scorer
+```
+
+**Minimal Example:**
+> ⚠️ **Critical**: The prompt template below is simplified for illustration. **Only the complete prompt template in `test_scorer_hf.py` will properly activate the model's fine-tuned scoring capabilities.** Custom or simplified prompts will not achieve optimal results.
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import re
+
+def extract_score(text):
+    matches = re.findall(r'<score>([\d.]+)</score>', text)
+    return float(matches[-1]) if matches else 0.0
+
+tokenizer = AutoTokenizer.from_pretrained("meituan-longcat/UNO-Scorer-Qwen3-14B")
+model = AutoModelForCausalLM.from_pretrained(
+    "meituan-longcat/UNO-Scorer-Qwen3-14B",
+    torch_dtype="auto",
+    device_map="auto"
+)
+
+# Prepare scoring prompt
+question = "Which animal appears in the image?"
+reference = "Sub-question 1: Elephant, total score 10 points"
+response = "I see an elephant in the image."
+
+prompt = f"""Please score the model's response based on the reference answer.
+
+Question: {question}
+Reference Answer: {reference}
+Model Response: {response}
+
+Provide a step-by-step analysis and output the total score in <score></score> tags."""
+
+# Generate score
+messages = [{"role": "user", "content": prompt}]
+text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+inputs = tokenizer([text], return_tensors="pt").to(model.device)
+outputs = model.generate(**inputs, max_new_tokens=2048)
+result = tokenizer.decode(outputs[0], skip_special_tokens=True)
+
+score = extract_score(result)
+print(f"Score: {score}/10")
 ```
 
-We recommend using vLLM for inference as it offers significantly better efficiency compared to the standard HuggingFace approach. Please follow the steps below to set up the environment and run the inference script provided in our official repository [UNO-Bench](https://github.com/meituan-longcat/UNO-Bench).
 
+### 🔄 How It Works
 
-### 1. Clone the Repository
-First, clone the UNO-Bench repository:
+UNO-Scorer evaluates model responses through a structured process:
 
-```bash
-git clone https://github.com/meituan-longcat/UNO-Bench.git
-cd UNO-Bench/uno-eval
+1. **Information Organization**: Extracts question content, reference answer, model response, and scoring criteria
+2. **Question Type Classification**: Identifies the question type (multiple-choice, numerical, enumeration, yes/no, short-answer, or essay)
+3. **Detailed Comparison**: Compares model response against reference answer using type-specific criteria
+4. **Score Extraction**: Outputs final score in `<score>X</score>` format (where X is 0-10)
+
+### 📥 Input Format Requirements
+
+The model expects three key inputs:
+
+| Component | Description | Example |
+| :--- | :--- | :--- |
+| **Question** | The original question posed to the model | "Which animals appear in the image?" |
+| **Reference Answer** | Ground truth answer with point allocation (sum to 10) | `Sub-question 1: Elephant, total score 10 points` |
+| **Model Response** | The response from the model being evaluated | "I see an elephant in the image." |
+
+#### Reference Answer Formatting (Critical!)
+
+Since the model is trained primarily on Chinese corpora, **formatting reference answers in Chinese yields significantly better results**. However, English formatting is also supported.
+
+**For Single-Answer Questions:**
+```
+1. {Answer}, total score 10 points, focus only on final answer correctness
+1. {答案}，总分10分，无需关注推理过程，最终答案正确即可
+```
+**For Multi-Part Questions:**
+```
+1. {Sub-Answer A} ({X} points); 2. {Sub-Answer B} ({Y} points)
+1. {子答案A}（{X}分）; 2. {子答案B}（{Y}分）
+```
+
+**With Custom Scoring Criteria:**
+
+```
+1. {Answer}, total score 10 points, scoring criteria: {detailed criteria}
+1. {答案}，总分10分，评分标准：{详细标准}
 ```
 
-### 2. Install Dependencies
-Install the necessary Python libraries:
+### 📤 Output Format
 
+The model returns:
+
+- **Detailed Evaluation**: Step-by-step analysis for each sub-question
+- **Score Tag**: `<score>X</score>` where X ranges from 0 to 10
+
+Example output:
+```
+Sub-question 1:
+
+Question Content: How many apples are in the image?
+Reference Answer: 2
+Model Response: There are two appels.
+Points: 1 point
+Question Type: Numerical
+Comparison Process: The reference answer is "2" and the model response is "two". The numerical values are completely identical, with only the expression format differing. This meets the scoring standard for numerical questions.
+
+Scoring Explanation: Completely correct, awarded 10 point.
+
+<score>10</score>
+```
+
+### 📋 Complete Evaluation Example
+
+See `test_scorer_hf.py` for a full working example with multiple question types:
+
+- Multiple-choice questions
+- Yes/No questions
+- Open-ended questions
+- Multi-part questions
+
+Run the example:
 ```bash
-pip install -r requirement.txt
+python3 test_scorer_hf.py --model-name /path/to/UNO-Scorer
 ```
 
-### 3. Run Inference
-We provide an example script based on **vLLM** for efficient model inference. You can run the following command to test the scorer:
+### 🚀 Optimized Inference with vLLM (Recommended for Production)
+
+For large-scale evaluation tasks, we strongly recommend using **vLLM** for significant performance improvements:
 
 ```bash
+# 1. Clone the repository
+git clone https://github.com/meituan-longcat/UNO-Bench.git
+cd UNO-Bench/uno-eval
+
+# 2. Install dependencies
+pip install -r requirements.txt
+
+# 3. Run vLLM-based inference
 bash examples/test_scorer_vllm.sh
 ```
 
-### 4. Adapt Your Reference Answer
-The most critical aspect of utilizing the UNO-Scorer lies in the proper formatting of the Reference Answer. Specifically, it is required to:
+**Why vLLM?**
+
+- **10-20x faster** inference compared to standard HuggingFace
+- Better batching support for multiple evaluation tasks
+- Lower memory footprint
+- Optimized for production deployments
 
-1. Assign point values to the answer components. The total points for the question should typically sum to 10 points.
-2. You may customize detailed scoring criteria for each reference answer to suit your needs(e.g., clarifying how to judge cases where the final choice is correct but the reasoning is flawed).
+### ⚠️ Important Notes
 
-Note: Since the model is primarily trained on Chinese corpora, it adheres more accurately to instructions when these specific descriptions are written in Chinese.
+1. **Language**: Chinese formatting in reference answers produces significantly better results due to the model's training data composition
+2. **Point Allocation**: Reference answers must have total points equal to 10
+3. **Score Extraction**: Always look for `<score>X</score>` in the output
+4. **Batch Processing**: Use vLLM for evaluating multiple responses efficiently
+5. **Question Type Awareness**: Ensure reference answers clearly specify the question type for optimal scoring
 
-You can structure the Reference Answer as follows:
+## 🎯 Supported Question Types
 
-| Question Type | Scenario | **Reference Answer** | Example |
-| :--- | :--- | :--- | :--- |
-| **Single Question** | The model only needs to check if the final result matches. | Format as a single sub-question (Sub-question 1) worth exactly 10 points.<br><br>Template:<br>`小问1：{Answer}，总分10分，无需关注推理过程，最终答案正确即可` | **Raw Answer:** "C"<br>**Input Answer:** `小问1：C，总分10分，无需关注推理过程，最终答案正确即可` |
-| **Multiple Question** | The model needs to grade specific checkpoints. | Break down the answer into numbered sub-steps with assigned points (summing to exactly 10).<br><br>Template:<br>`1. {Sub-Answer A} ({X} points); 2. {Sub-Answer B} ({Y} points).` | **Raw Answer:** "5 apples, 6 bananas"<br>**Input Answer:** `1. 5 apples (4 points); 2. 6 bananas (6 points).` |
+UNO-Scorer supports evaluation across 6 distinct question types:
 
+| Question Type | Description | Scoring Rule |
+| :--- | :--- | :--- |
+| **Multiple-Choice** | Select correct option from given choices | Response must match the correct option exactly |
+| **Numerical** | Provide specific numerical values | No tolerance for numerical errors |
+| **Enumeration** | List all required items | Must include all items, no omissions or errors |
+| **Yes/No** | Binary judgment questions | Response judgment must match reference answer |
+| **Short-Answer** | Brief factual answers | Semantic equivalence acceptable, expression flexibility allowed |
+| **Essay** | Longer analytical responses | Must contain core viewpoints from reference answer |
 
 ## 📜 Citation
 
@@ -111,6 +240,18 @@ If you find this model or the UNO-Bench useful for your research, please cite ou
 }
 ```
 
----
+## ⚖️ License & Disclaimer
+
+This model is released under the **Apache 2.0 License**. It is based on Qwen3-14B. Please strictly follow the license and usage policy of the original Qwen model series.
+
+**Disclaimer**: This model is designed for research and evaluation purposes. Users are responsible for ensuring their use complies with applicable laws and regulations.
+
+## 🤝 Contributing
+
+We welcome contributions and feedback! Please feel free to:
+- Report issues or bugs
+- Suggest improvements
+- Share your evaluation results
+- Contribute enhancements
 
-**Disclaimer:** This model is based on Qwen3-14B. Please strictly follow the license and usage policy of the original Qwen model series.
+For more information, visit our [GitHub repository](https://github.com/meituan-longcat/UNO-Bench).