File size: 6,799 Bytes
67aa874 98d57cb 67aa874 6c9cc48 67aa874 2047d77 125e3d7 67aa874 2047d77 67aa874 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 |
---
license: apache-2.0
language:
- en
tags:
- reinforcement-learning
- teacher-student
- adaptive-learning
- pedagogy
- rlhf
- rlaif
base_model: Qwen/Qwen3-8B
model_type: qwen3
datasets:
- Arc-Intelligence/Arc-ATLAS-Teach-v0
model-index:
- name: ATLAS-8B-Thinking
results:
- task:
type: text-generation
name: Reinforcement Learning Teaching
dataset:
name: Arc-Intelligence/Arc-ATLAS-Teach-v0
type: Arc-Intelligence/Arc-ATLAS-Teach-v0
metrics:
- name: Non-Degradation Rate
value: 97%
type: non_degradation_rate
- name: Average Accuracy Improvement
value: +15.7%
type: average_accuracy_improvement
- name: Task Completion Rate Improvement
value: +31.2%
type: task_completion_rate_improvement
- name: Response Token Reduction
value: '-37.2%'
type: response_token_reduction
pipeline_tag: text-generation
library_name: transformers
---
# ATLAS-8B-Thinking
**ATLAS-8B-Thinking** is a specialized teacher model developed by Arc Intelligence, designed to solve the core reliability problem in reinforcement learning for LLMs. Standard RL fine-tuning is often brittle, leading to performance degradation where new skills are learned at the expense of old ones.
This model reframes the training process as one of **effective pedagogy**. Instead of just optimizing a student model, `ATLAS-8B-Thinking` first uses a lightweight **diagnostic probe** to assess the student's reasoning. Based on this diagnosis, it provides **adaptive guidance**—comprehensive help for struggling models and minimal intervention for capable ones. This "do no harm" approach ensures consistent capability improvement without the usual side effects of RL.
This model is a core component of the open-source [ATLAS Framework](https://github.com/Arc-Computer/ATLAS) and is designed to train and improve other language models.
## Model Performance
The ATLAS framework, using this teacher model, produces the following improvements in a student model (Qwen3-4B) compared to the student baseline. The results highlight a rare combination of increased performance, higher efficiency, and fundamental reliability.
| Metric | Improvement | Notes |
| ---------------------- | ----------- | ---------------------------------------------------------- |
| **Non-Degradation Rate** | **97%** | Core metric showing reliability and avoidance of skill loss. |
| Average Accuracy | +15.7% | Across the Arc-ATLAS-Teach-v0 evaluation set. |
| Task Completion Rate | +31.2% | Student model completes tasks it previously failed. |
| Response Tokens | -37.2% | More efficient and concise reasoning. |
## How to Use
`ATLAS-8B-Thinking` is not a standard instruction-tuned model for direct chat. It is a core component of the ATLAS training framework, designed to interact with a "student" model in a two-pass process.
### Conceptual Usage
The following is a simplified, conceptual example of the ATLAS interaction loop. The full implementation is available in the official repository.
```python
# A conceptual example of the ATLAS interaction loop
from transformers import AutoModelForCausalLM, AutoTokenizer
# Load the teacher and a student model
teacher_model = AutoModelForCausalLM.from_pretrained("Arc-Intelligence/ATLAS-8B-Thinking")
student_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B") # The model to be improved
problem = "A farmer has 52 trees planted in a row over a length of 1850 meters. What is the distance between each tree?"
# 1. Teacher creates a diagnostic probe to assess the student's initial approach
# This step is abstracted in the actual framework
diagnostic_probe = "To find the distance between the trees, what is the first critical calculation you would make?"
# 2. Student responds to the probe
# (Implementation detail: you would get the student's response here)
student_reasoning_trace = "I would divide the total length (1850m) by the number of trees (52)."
# 3. Teacher assesses the trace and provides adaptive guidance
# The teacher recognizes this common off-by-one error.
# (Implementation detail: the teacher model generates this guidance)
adaptive_guidance = "Your approach is close. Remember that 52 trees create 51 intervals between them. The distance is uniform across these intervals."
# 4. The student uses the guidance to solve the problem
final_prompt = problem + "\n" + adaptive_guidance
# (Implementation detail: the student model generates the final answer)
final_answer = "1850 meters / 51 intervals = 36.27 meters per interval."
```
### Running the Full Training Pipeline
To replicate our results or train your own models using the ATLAS framework, clone the official repository and follow the setup instructions.
```bash
# 1. Clone the repository
git clone [https://github.com/Arc-Computer/ATLAS](https://github.com/Arc-Computer/ATLAS)
cd ATLAS
# 2. Install dependencies
bash scripts/install_py312.sh
# 3. Run training
# Phase 1: Supervised Fine-Tuning (SFT)
scripts/launch.sh 4 configs/run/teacher_sft.yaml
# Phase 2: Reinforcement Learning (RL)
scripts/launch_with_server.sh 1 3 configs/run/teacher_rcl.yaml
```
## Training Details
- **Base Model:** [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B)
- **Training Framework:** ATLAS (SFT → RL with GRPO)
- **Key Feature:** The RL phase uses an asymmetric reward function that heavily penalizes any instance of student performance degradation, which is key to the framework's reliability.
- **Dataset:** [Arc-Intelligence/Arc-ATLAS-Teach-v0](https://huggingface.co/datasets/Arc-Intelligence/Arc-ATLAS-Teach-v0)
- **Context Length:** 8192 tokens
## Citation
If you use the ATLAS framework or our models in your research, please cite our work:
```bibtex
@misc{barnes2025atlas,
title={{ATLAS: Adaptive Teaching and Learning Alignment System for Reinforcement Learning}},
author={Jarrod Barnes and Aman Jaglan},
year={2025},
publisher={Arc Intelligence},
note={Technical Report},
url={[https://github.com/Arc-Computer/ATLAS](https://github.com/Arc-Computer/ATLAS)}
}
```
## Project Resources
- **GitHub Repository:** [https://github.com/Arc-Computer/ATLAS](https://github.com/Arc-Computer/ATLAS)
- **Companion Model:** [ATLAS-8B-Instruct](https://huggingface.co/Arc-Intelligence/ATLAS-8B-Instruct)
- **Training Dataset:** [Arc-ATLAS-Teach-v0](https://huggingface.co/datasets/Arc-Intelligence/Arc-ATLAS-Teach-v0) |