--- library_name: transformers pipeline_tag: image-text-to-text license: apache-2.0 --- # PaCo-Reward-7B: A Pairwise Consistency Evaluator from the PaCo-RL Framework
       
This repository contains **PaCo-Reward-7B**, a key component of the **PaCo-RL** framework, as presented in the paper: [**PaCo-RL: Advancing Reinforcement Learning for Consistent Image Generation with Pairwise Reward Modeling**](https://huggingface.co/papers/2512.04784) The **PaCo-RL** framework is designed for consistent image generation through reinforcement learning, aiming to preserve identities, styles, and logical coherence across multiple images for applications like storytelling and character design. **PaCo-Reward-7B** specifically acts as a pairwise consistency evaluator. It is trained on a large-scale dataset constructed via automated sub-figure pairing and evaluates consistency through a generative, autoregressive scoring mechanism, enhanced by task-aware instructions and Chain-of-Thought (CoT) reasoning. - **Project Page:** https://x-gengroup.github.io/HomePage_PaCo-RL/ - **Code Repository:** https://github.com/X-GenGroup/PaCo-RL ## 🌟 Overview **PaCo-RL** is a comprehensive framework for consistent image generation through reinforcement learning, addressing challenges in preserving identities, styles, and logical coherence across multiple images for storytelling and character design applications. ### Key Components - **PaCo-Reward**: A pairwise consistency evaluator with task-aware instruction and CoT reasoning. - **PaCo-GRPO**: Efficient RL optimization with resolution-decoupled training and log-tamed multi-reward aggregation
PaCo-RL Overview
## Example Usage PaCo-Reward-7B is fine-tuned based on [Qwen/Qwen2.5-VL-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct), so you can load the model similarly with the following code: ```python import torch from transformers import Qwen2_5_VLForConditionalGeneration, AutoTokenizer, AutoProcessor from qwen_vl_utils import process_vision_info model = Qwen2_5_VLForConditionalGeneration.from_pretrained( "X-GenGroup/PaCo-Reward-7B", torch_dtype="bfloat16", device_map="auto" ) # default processer processor = AutoProcessor.from_pretrained("X-GenGroup/PaCo-Reward-7B") image1 = 'https://huggingface.co/X-GenGroup/PaCo-Reward-7B/resolve/main/images/image_1.jpg' image2 = 'https://huggingface.co/X-GenGroup/PaCo-Reward-7B/resolve/main/images/image_2.jpg' main_prompt = 'Generate multiple images portraying a medical scene of a dentist in scrubs. The images should include activities such as explaining oral hygiene to a patient, taking X-rays of teeth, cleaning teeth in a dental office, and filling a cavity during an appointment. The setting should depict a realistic dental clinic.' text_prompt = ( f"Given two subfigures generated based on the theme: \"{main_prompt}\", " f"do the two images maintain consistency in terms of style, logic and identity? " f"Answer \"Yes\" and \"No\" first, and then provide detailed reasons." ) # Example: Compare whether two images are visually consistent messages_1 = [ { "role": "user", "content": [ {"type": "image", "image": image1}, {"type": "image", "image": image2}, {"type": "text", "text": text_prompt}, ], } ] # Preparation for inference text = processor.apply_chat_template( messages_1, tokenize=False, add_generation_prompt=True ) image_inputs, video_inputs = process_vision_info(messages_1) inputs = processor( text=[text], images=image_inputs, videos=video_inputs, padding=True, return_tensors="pt", ) inputs = inputs.to("cuda") # Inference: Calculate consistency score # Get logits for first token with torch.no_grad(): outputs = model(**inputs) first_token_logits = outputs.logits[0, -1, :] # Last position of prompt # Get token IDs for "Yes" and "No" yes_id = processor.tokenizer.encode("Yes", add_special_tokens=False)[0] no_id = processor.tokenizer.encode("No", add_special_tokens=False)[0] # Calculate probability yes_logit = first_token_logits[yes_id] no_logit = first_token_logits[no_id] yes_prob = torch.exp(yes_logit) / (torch.exp(yes_logit) + torch.exp(no_logit)) print(f"Consistency Score (Yes Conditional Probability): {yes_prob.item():.4f}") # Inference: Generate detailed reasons generated_ids = model.generate(**inputs, max_new_tokens=512) generated_ids_trimmed = [ out_ids[len(in_ids) :] for in_ids, out_ids in zip(inputs.input_ids, generated_ids) ] output_text = processor.batch_decode( generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False ) print(output_text[0]) ``` ## 🎁 Model Zoo This model is part of a larger collection of models within the PaCo-RL framework. More related models can be found in the [PaCo-RL Hugging Face collection](https://huggingface.co/collections/X-GenGroup/paco-rl). | Model | Type | HuggingFace | | :---------------------- | :------------------ | :--------------------------------------------------------- | | **PaCo-Reward-7B** | Reward Model | [🤗 Link](https://huggingface.co/X-GenGroup/PaCo-Reward-7B) | | **PaCo-Reward-7B-Lora** | Reward Model (LoRA) | [🤗 Link](https://huggingface.co/X-GenGroup/PaCo-Reward-7B-Lora) | | **PaCo-FLUX.1-dev** | T2I Model (LoRA) | [🤗 Link](https://huggingface.co/X-GenGroup/PaCo-FLUX.1-dev-Lora) | | **PaCo-FLUX.1-Kontext-dev** | Image Editing Model (LoRA) | [🤗 Link](https://huggingface.co/X-GenGroup/PaCo-FLUX.1-Kontext-Lora) | | **PaCo-QwenImage-Edit** | Image Editing Model (LoRA) | [🤗 Link](https://huggingface.co/X-GenGroup/PaCo-Qwen-Image-Edit-Lora) | ## ⭐ Citation If you find our work helpful or inspiring, please feel free to cite it: ```bibtex @misc{ping2025pacorladvancingreinforcementlearning, title={PaCo-RL: Advancing Reinforcement Learning for Consistent Image Generation with Pairwise Reward Modeling}, author={Bowen Ping and Chengyou Jia and Minnan Luo and Changliang Xia and Xin Shen and Zhuohang Dang and Hangwei Qian}, year={2025}, eprint={2512.04784}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2512.04784}, } ```