Add files using upload-large-folder tool

Qwen3-30B-A3B-Instruct-2507-sft-fusang-swa=2k_sink=0/checkpoint-870/README.md

@@ -0,0 +1,207 @@
+---
+base_model: /share/models/Qwen3-30B-A3B-Instruct-2507
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/share/models/Qwen3-30B-A3B-Instruct-2507
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1

Qwen3-30B-A3B-Instruct-2507-sft-fusang-swa=2k_sink=0/checkpoint-870/chat_template.jinja

@@ -0,0 +1,61 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if message.content is string %}
+        {%- set content = message.content %}
+    {%- else %}
+        {%- set content = '' %}
+    {%- endif %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

Qwen3-30B-A3B-Instruct-2507-sft-fusang-swa=2k_sink=0_falayer=25_from0_step2_fadec/checkpoint-994/added_tokens.json

@@ -0,0 +1,28 @@
+{
+  "</think>": 151668,
+  "</tool_call>": 151658,
+  "</tool_response>": 151666,
+  "<think>": 151667,
+  "<tool_call>": 151657,
+  "<tool_response>": 151665,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

Qwen3-30B-A3B-Instruct-2507-sft-fusang-swa=2k_sink=0_falayer=25_from0_step2_fadec/checkpoint-994/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Qwen3-30B-A3B-Instruct-2507-sft-fusang-swa=2k_sink=0_falayer=25_from0_step2_fadec/checkpoint-994/tokenizer_config.json

@@ -0,0 +1,239 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151665": {
+      "content": "<tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151666": {
+      "content": "</tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151667": {
+      "content": "<think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151668": {
+      "content": "</think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 1010000,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

Qwen3-30B-A3B-Instruct-2507-sft-fusang-swa=2k_sink=100_fadec/checkpoint-870/added_tokens.json

@@ -0,0 +1,28 @@
+{
+  "</think>": 151668,
+  "</tool_call>": 151658,
+  "</tool_response>": 151666,
+  "<think>": 151667,
+  "<tool_call>": 151657,
+  "<tool_response>": 151665,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

Qwen3-30B-A3B-Instruct-2507-sft-fusang-swa=2k_sink=100_falayer=25_from0_step2_fadec/checkpoint-870/adapter_config.json

@@ -0,0 +1,38 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/share/models/Qwen3-30B-A3B-Instruct-2507",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "v_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": true
+}

Qwen3-30B-A3B-Instruct-2507-sft-fusang-swa=2k_sink=100_falayer=25_from0_step2_fadec/checkpoint-870/added_tokens.json

@@ -0,0 +1,28 @@
+{
+  "</think>": 151668,
+  "</tool_call>": 151658,
+  "</tool_response>": 151666,
+  "<think>": 151667,
+  "<tool_call>": 151657,
+  "<tool_response>": 151665,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

Qwen3-30B-A3B-Instruct-2507-sft-fusang-swa=2k_sink=100_falayer=25_from0_step2_fadec/checkpoint-870/chat_template.jinja

@@ -0,0 +1,61 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if message.content is string %}
+        {%- set content = message.content %}
+    {%- else %}
+        {%- set content = '' %}
+    {%- endif %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

Qwen3-30B-A3B-Thinking-2507-sft-fusang-swa=2k_sink=0/checkpoint-862/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Qwen3-30B-A3B-Thinking-2507-sft-fusang-swa=2k_sink=0/checkpoint-862/trainer_state.json

@@ -0,0 +1,1238 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.0,
+  "eval_steps": 500,
+  "global_step": 862,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.005803830528148578,
+      "grad_norm": 0.4036159813404083,
+      "learning_rate": 9.090909090909091e-06,
+      "loss": 0.8451,
+      "step": 5
+    },
+    {
+      "epoch": 0.011607661056297156,
+      "grad_norm": 0.9483785033226013,
+      "learning_rate": 2.0454545454545457e-05,
+      "loss": 0.8263,
+      "step": 10
+    },
+    {
+      "epoch": 0.017411491584445733,
+      "grad_norm": 0.4712241590023041,
+      "learning_rate": 3.181818181818182e-05,
+      "loss": 0.7207,
+      "step": 15
+    },
+    {
+      "epoch": 0.023215322112594312,
+      "grad_norm": 0.33003488183021545,
+      "learning_rate": 4.318181818181819e-05,
+      "loss": 0.6433,
+      "step": 20
+    },
+    {
+      "epoch": 0.02901915264074289,
+      "grad_norm": 0.16803228855133057,
+      "learning_rate": 5.4545454545454546e-05,
+      "loss": 0.5775,
+      "step": 25
+    },
+    {
+      "epoch": 0.034822983168891465,
+      "grad_norm": 0.1392829865217209,
+      "learning_rate": 6.59090909090909e-05,
+      "loss": 0.5424,
+      "step": 30
+    },
+    {
+      "epoch": 0.04062681369704005,
+      "grad_norm": 0.10901425778865814,
+      "learning_rate": 7.727272727272727e-05,
+      "loss": 0.5209,
+      "step": 35
+    },
+    {
+      "epoch": 0.046430644225188625,
+      "grad_norm": 0.08062315732240677,
+      "learning_rate": 8.863636363636364e-05,
+      "loss": 0.5258,
+      "step": 40
+    },
+    {
+      "epoch": 0.0522344747533372,
+      "grad_norm": 0.12822897732257843,
+      "learning_rate": 0.0001,
+      "loss": 0.5198,
+      "step": 45
+    },
+    {
+      "epoch": 0.05803830528148578,
+      "grad_norm": 0.09056111425161362,
+      "learning_rate": 9.938875305623473e-05,
+      "loss": 0.519,
+      "step": 50
+    },
+    {
+      "epoch": 0.06384213580963435,
+      "grad_norm": 0.07849844545125961,
+      "learning_rate": 9.877750611246945e-05,
+      "loss": 0.5106,
+      "step": 55
+    },
+    {
+      "epoch": 0.06964596633778293,
+      "grad_norm": 0.06696949154138565,
+      "learning_rate": 9.816625916870417e-05,
+      "loss": 0.509,
+      "step": 60
+    },
+    {
+      "epoch": 0.07544979686593152,
+      "grad_norm": 0.06861168146133423,
+      "learning_rate": 9.755501222493889e-05,
+      "loss": 0.4936,
+      "step": 65
+    },
+    {
+      "epoch": 0.0812536273940801,
+      "grad_norm": 0.06149851903319359,
+      "learning_rate": 9.69437652811736e-05,
+      "loss": 0.484,
+      "step": 70
+    },
+    {
+      "epoch": 0.08705745792222867,
+      "grad_norm": 0.06428489834070206,
+      "learning_rate": 9.633251833740831e-05,
+      "loss": 0.5211,
+      "step": 75
+    },
+    {
+      "epoch": 0.09286128845037725,
+      "grad_norm": 0.0690334141254425,
+      "learning_rate": 9.572127139364305e-05,
+      "loss": 0.4749,
+      "step": 80
+    },
+    {
+      "epoch": 0.09866511897852583,
+      "grad_norm": 0.0862906351685524,
+      "learning_rate": 9.511002444987775e-05,
+      "loss": 0.476,
+      "step": 85
+    },
+    {
+      "epoch": 0.1044689495066744,
+      "grad_norm": 0.07444003969430923,
+      "learning_rate": 9.449877750611247e-05,
+      "loss": 0.4811,
+      "step": 90
+    },
+    {
+      "epoch": 0.11027278003482298,
+      "grad_norm": 0.06518575549125671,
+      "learning_rate": 9.38875305623472e-05,
+      "loss": 0.4798,
+      "step": 95
+    },
+    {
+      "epoch": 0.11607661056297155,
+      "grad_norm": 0.07294116169214249,
+      "learning_rate": 9.327628361858191e-05,
+      "loss": 0.4913,
+      "step": 100
+    },
+    {
+      "epoch": 0.12188044109112015,
+      "grad_norm": 0.06533044576644897,
+      "learning_rate": 9.266503667481663e-05,
+      "loss": 0.4795,
+      "step": 105
+    },
+    {
+      "epoch": 0.1276842716192687,
+      "grad_norm": 0.07440106570720673,
+      "learning_rate": 9.205378973105135e-05,
+      "loss": 0.4785,
+      "step": 110
+    },
+    {
+      "epoch": 0.1334881021474173,
+      "grad_norm": 0.05704384669661522,
+      "learning_rate": 9.144254278728606e-05,
+      "loss": 0.4891,
+      "step": 115
+    },
+    {
+      "epoch": 0.13929193267556586,
+      "grad_norm": 0.0678258091211319,
+      "learning_rate": 9.08312958435208e-05,
+      "loss": 0.4954,
+      "step": 120
+    },
+    {
+      "epoch": 0.14509576320371445,
+      "grad_norm": 0.06707849353551865,
+      "learning_rate": 9.02200488997555e-05,
+      "loss": 0.4693,
+      "step": 125
+    },
+    {
+      "epoch": 0.15089959373186304,
+      "grad_norm": 0.07949420809745789,
+      "learning_rate": 8.960880195599024e-05,
+      "loss": 0.4828,
+      "step": 130
+    },
+    {
+      "epoch": 0.1567034242600116,
+      "grad_norm": 0.055451974272727966,
+      "learning_rate": 8.899755501222494e-05,
+      "loss": 0.4906,
+      "step": 135
+    },
+    {
+      "epoch": 0.1625072547881602,
+      "grad_norm": 0.05859282240271568,
+      "learning_rate": 8.838630806845966e-05,
+      "loss": 0.5034,
+      "step": 140
+    },
+    {
+      "epoch": 0.16831108531630876,
+      "grad_norm": 0.06981738656759262,
+      "learning_rate": 8.777506112469438e-05,
+      "loss": 0.4946,
+      "step": 145
+    },
+    {
+      "epoch": 0.17411491584445735,
+      "grad_norm": 0.06059090420603752,
+      "learning_rate": 8.71638141809291e-05,
+      "loss": 0.4827,
+      "step": 150
+    },
+    {
+      "epoch": 0.1799187463726059,
+      "grad_norm": 0.055644381791353226,
+      "learning_rate": 8.655256723716382e-05,
+      "loss": 0.4607,
+      "step": 155
+    },
+    {
+      "epoch": 0.1857225769007545,
+      "grad_norm": 0.055599577724933624,
+      "learning_rate": 8.594132029339854e-05,
+      "loss": 0.4806,
+      "step": 160
+    },
+    {
+      "epoch": 0.1915264074289031,
+      "grad_norm": 0.065301813185215,
+      "learning_rate": 8.533007334963325e-05,
+      "loss": 0.4804,
+      "step": 165
+    },
+    {
+      "epoch": 0.19733023795705165,
+      "grad_norm": 0.07100919634103775,
+      "learning_rate": 8.471882640586798e-05,
+      "loss": 0.4788,
+      "step": 170
+    },
+    {
+      "epoch": 0.20313406848520024,
+      "grad_norm": 0.05895709618926048,
+      "learning_rate": 8.410757946210269e-05,
+      "loss": 0.483,
+      "step": 175
+    },
+    {
+      "epoch": 0.2089378990133488,
+      "grad_norm": 0.060625575482845306,
+      "learning_rate": 8.349633251833741e-05,
+      "loss": 0.4856,
+      "step": 180
+    },
+    {
+      "epoch": 0.2147417295414974,
+      "grad_norm": 0.060117509216070175,
+      "learning_rate": 8.288508557457213e-05,
+      "loss": 0.4852,
+      "step": 185
+    },
+    {
+      "epoch": 0.22054556006964596,
+      "grad_norm": 0.04944780096411705,
+      "learning_rate": 8.227383863080685e-05,
+      "loss": 0.4762,
+      "step": 190
+    },
+    {
+      "epoch": 0.22634939059779455,
+      "grad_norm": 0.05715511366724968,
+      "learning_rate": 8.166259168704157e-05,
+      "loss": 0.4534,
+      "step": 195
+    },
+    {
+      "epoch": 0.2321532211259431,
+      "grad_norm": 0.05923309177160263,
+      "learning_rate": 8.105134474327629e-05,
+      "loss": 0.4715,
+      "step": 200
+    },
+    {
+      "epoch": 0.2379570516540917,
+      "grad_norm": 0.05913982540369034,
+      "learning_rate": 8.044009779951101e-05,
+      "loss": 0.4672,
+      "step": 205
+    },
+    {
+      "epoch": 0.2437608821822403,
+      "grad_norm": 0.05672944709658623,
+      "learning_rate": 7.982885085574573e-05,
+      "loss": 0.5026,
+      "step": 210
+    },
+    {
+      "epoch": 0.24956471271038885,
+      "grad_norm": 0.07731504738330841,
+      "learning_rate": 7.921760391198044e-05,
+      "loss": 0.4734,
+      "step": 215
+    },
+    {
+      "epoch": 0.2553685432385374,
+      "grad_norm": 0.05067066475749016,
+      "learning_rate": 7.860635696821517e-05,
+      "loss": 0.4598,
+      "step": 220
+    },
+    {
+      "epoch": 0.261172373766686,
+      "grad_norm": 0.05259395018219948,
+      "learning_rate": 7.799511002444988e-05,
+      "loss": 0.4802,
+      "step": 225
+    },
+    {
+      "epoch": 0.2669762042948346,
+      "grad_norm": 0.06773041188716888,
+      "learning_rate": 7.73838630806846e-05,
+      "loss": 0.4715,
+      "step": 230
+    },
+    {
+      "epoch": 0.2727800348229832,
+      "grad_norm": 0.05656243488192558,
+      "learning_rate": 7.677261613691932e-05,
+      "loss": 0.4955,
+      "step": 235
+    },
+    {
+      "epoch": 0.2785838653511317,
+      "grad_norm": 0.0642087385058403,
+      "learning_rate": 7.616136919315404e-05,
+      "loss": 0.4801,
+      "step": 240
+    },
+    {
+      "epoch": 0.2843876958792803,
+      "grad_norm": 0.07234173268079758,
+      "learning_rate": 7.555012224938876e-05,
+      "loss": 0.4702,
+      "step": 245
+    },
+    {
+      "epoch": 0.2901915264074289,
+      "grad_norm": 0.05371543765068054,
+      "learning_rate": 7.493887530562348e-05,
+      "loss": 0.4949,
+      "step": 250
+    },
+    {
+      "epoch": 0.2959953569355775,
+      "grad_norm": 0.06422676146030426,
+      "learning_rate": 7.432762836185819e-05,
+      "loss": 0.4731,
+      "step": 255
+    },
+    {
+      "epoch": 0.3017991874637261,
+      "grad_norm": 0.06138024479150772,
+      "learning_rate": 7.371638141809292e-05,
+      "loss": 0.4947,
+      "step": 260
+    },
+    {
+      "epoch": 0.3076030179918746,
+      "grad_norm": 0.05534062162041664,
+      "learning_rate": 7.310513447432763e-05,
+      "loss": 0.4614,
+      "step": 265
+    },
+    {
+      "epoch": 0.3134068485200232,
+      "grad_norm": 0.053715579211711884,
+      "learning_rate": 7.249388753056235e-05,
+      "loss": 0.4634,
+      "step": 270
+    },
+    {
+      "epoch": 0.3192106790481718,
+      "grad_norm": 0.05673748627305031,
+      "learning_rate": 7.188264058679707e-05,
+      "loss": 0.4783,
+      "step": 275
+    },
+    {
+      "epoch": 0.3250145095763204,
+      "grad_norm": 0.05728138983249664,
+      "learning_rate": 7.127139364303179e-05,
+      "loss": 0.4845,
+      "step": 280
+    },
+    {
+      "epoch": 0.3308183401044689,
+      "grad_norm": 0.05750945582985878,
+      "learning_rate": 7.066014669926651e-05,
+      "loss": 0.4945,
+      "step": 285
+    },
+    {
+      "epoch": 0.3366221706326175,
+      "grad_norm": 0.1356944888830185,
+      "learning_rate": 7.004889975550123e-05,
+      "loss": 0.4726,
+      "step": 290
+    },
+    {
+      "epoch": 0.3424260011607661,
+      "grad_norm": 0.061065953224897385,
+      "learning_rate": 6.943765281173595e-05,
+      "loss": 0.4752,
+      "step": 295
+    },
+    {
+      "epoch": 0.3482298316889147,
+      "grad_norm": 0.06057751178741455,
+      "learning_rate": 6.882640586797067e-05,
+      "loss": 0.4778,
+      "step": 300
+    },
+    {
+      "epoch": 0.3540336622170633,
+      "grad_norm": 0.05359559506177902,
+      "learning_rate": 6.821515892420538e-05,
+      "loss": 0.4622,
+      "step": 305
+    },
+    {
+      "epoch": 0.3598374927452118,
+      "grad_norm": 0.07310923933982849,
+      "learning_rate": 6.760391198044011e-05,
+      "loss": 0.4894,
+      "step": 310
+    },
+    {
+      "epoch": 0.3656413232733604,
+      "grad_norm": 0.0615098662674427,
+      "learning_rate": 6.699266503667482e-05,
+      "loss": 0.4736,
+      "step": 315
+    },
+    {
+      "epoch": 0.371445153801509,
+      "grad_norm": 0.05349689722061157,
+      "learning_rate": 6.638141809290954e-05,
+      "loss": 0.4642,
+      "step": 320
+    },
+    {
+      "epoch": 0.3772489843296576,
+      "grad_norm": 0.05894140899181366,
+      "learning_rate": 6.577017114914426e-05,
+      "loss": 0.4894,
+      "step": 325
+    },
+    {
+      "epoch": 0.3830528148578062,
+      "grad_norm": 0.061526838690042496,
+      "learning_rate": 6.515892420537898e-05,
+      "loss": 0.4657,
+      "step": 330
+    },
+    {
+      "epoch": 0.3888566453859547,
+      "grad_norm": 0.059954240918159485,
+      "learning_rate": 6.45476772616137e-05,
+      "loss": 0.4777,
+      "step": 335
+    },
+    {
+      "epoch": 0.3946604759141033,
+      "grad_norm": 0.07039766758680344,
+      "learning_rate": 6.393643031784842e-05,
+      "loss": 0.4682,
+      "step": 340
+    },
+    {
+      "epoch": 0.4004643064422519,
+      "grad_norm": 0.05843193084001541,
+      "learning_rate": 6.332518337408312e-05,
+      "loss": 0.4785,
+      "step": 345
+    },
+    {
+      "epoch": 0.4062681369704005,
+      "grad_norm": 0.05405830591917038,
+      "learning_rate": 6.271393643031786e-05,
+      "loss": 0.4558,
+      "step": 350
+    },
+    {
+      "epoch": 0.412071967498549,
+      "grad_norm": 0.05697088688611984,
+      "learning_rate": 6.210268948655256e-05,
+      "loss": 0.4705,
+      "step": 355
+    },
+    {
+      "epoch": 0.4178757980266976,
+      "grad_norm": 0.07413748651742935,
+      "learning_rate": 6.14914425427873e-05,
+      "loss": 0.4942,
+      "step": 360
+    },
+    {
+      "epoch": 0.4236796285548462,
+      "grad_norm": 0.05946504697203636,
+      "learning_rate": 6.0880195599022005e-05,
+      "loss": 0.4667,
+      "step": 365
+    },
+    {
+      "epoch": 0.4294834590829948,
+      "grad_norm": 0.056219637393951416,
+      "learning_rate": 6.026894865525673e-05,
+      "loss": 0.4601,
+      "step": 370
+    },
+    {
+      "epoch": 0.4352872896111434,
+      "grad_norm": 0.06177263706922531,
+      "learning_rate": 5.9657701711491446e-05,
+      "loss": 0.4903,
+      "step": 375
+    },
+    {
+      "epoch": 0.4410911201392919,
+      "grad_norm": 0.05127112567424774,
+      "learning_rate": 5.9046454767726166e-05,
+      "loss": 0.4593,
+      "step": 380
+    },
+    {
+      "epoch": 0.4468949506674405,
+      "grad_norm": 0.06899844110012054,
+      "learning_rate": 5.843520782396088e-05,
+      "loss": 0.4512,
+      "step": 385
+    },
+    {
+      "epoch": 0.4526987811955891,
+      "grad_norm": 0.06159770116209984,
+      "learning_rate": 5.7823960880195606e-05,
+      "loss": 0.4741,
+      "step": 390
+    },
+    {
+      "epoch": 0.4585026117237377,
+      "grad_norm": 0.057136863470077515,
+      "learning_rate": 5.721271393643032e-05,
+      "loss": 0.4611,
+      "step": 395
+    },
+    {
+      "epoch": 0.4643064422518862,
+      "grad_norm": 0.056901607662439346,
+      "learning_rate": 5.660146699266504e-05,
+      "loss": 0.4695,
+      "step": 400
+    },
+    {
+      "epoch": 0.4701102727800348,
+      "grad_norm": 0.058210041373968124,
+      "learning_rate": 5.5990220048899754e-05,
+      "loss": 0.4677,
+      "step": 405
+    },
+    {
+      "epoch": 0.4759141033081834,
+      "grad_norm": 0.05751234292984009,
+      "learning_rate": 5.537897310513448e-05,
+      "loss": 0.4532,
+      "step": 410
+    },
+    {
+      "epoch": 0.481717933836332,
+      "grad_norm": 0.049704063683748245,
+      "learning_rate": 5.4767726161369194e-05,
+      "loss": 0.4743,
+      "step": 415
+    },
+    {
+      "epoch": 0.4875217643644806,
+      "grad_norm": 0.05579414218664169,
+      "learning_rate": 5.4156479217603914e-05,
+      "loss": 0.4664,
+      "step": 420
+    },
+    {
+      "epoch": 0.4933255948926291,
+      "grad_norm": 0.054620131850242615,
+      "learning_rate": 5.354523227383863e-05,
+      "loss": 0.4727,
+      "step": 425
+    },
+    {
+      "epoch": 0.4991294254207777,
+      "grad_norm": 0.0595741868019104,
+      "learning_rate": 5.2933985330073355e-05,
+      "loss": 0.4614,
+      "step": 430
+    },
+    {
+      "epoch": 0.5049332559489262,
+      "grad_norm": 0.05558570474386215,
+      "learning_rate": 5.232273838630807e-05,
+      "loss": 0.4847,
+      "step": 435
+    },
+    {
+      "epoch": 0.5107370864770748,
+      "grad_norm": 0.05618005990982056,
+      "learning_rate": 5.1711491442542795e-05,
+      "loss": 0.4822,
+      "step": 440
+    },
+    {
+      "epoch": 0.5165409170052234,
+      "grad_norm": 0.05827624350786209,
+      "learning_rate": 5.110024449877751e-05,
+      "loss": 0.4821,
+      "step": 445
+    },
+    {
+      "epoch": 0.522344747533372,
+      "grad_norm": 0.056044381111860275,
+      "learning_rate": 5.048899755501223e-05,
+      "loss": 0.4905,
+      "step": 450
+    },
+    {
+      "epoch": 0.5281485780615206,
+      "grad_norm": 0.04992903396487236,
+      "learning_rate": 4.987775061124695e-05,
+      "loss": 0.4596,
+      "step": 455
+    },
+    {
+      "epoch": 0.5339524085896692,
+      "grad_norm": 0.06089325621724129,
+      "learning_rate": 4.926650366748167e-05,
+      "loss": 0.4769,
+      "step": 460
+    },
+    {
+      "epoch": 0.5397562391178178,
+      "grad_norm": 0.05053069069981575,
+      "learning_rate": 4.865525672371638e-05,
+      "loss": 0.4797,
+      "step": 465
+    },
+    {
+      "epoch": 0.5455600696459664,
+      "grad_norm": 0.06190374866127968,
+      "learning_rate": 4.80440097799511e-05,
+      "loss": 0.4812,
+      "step": 470
+    },
+    {
+      "epoch": 0.551363900174115,
+      "grad_norm": 0.05578217655420303,
+      "learning_rate": 4.743276283618582e-05,
+      "loss": 0.4553,
+      "step": 475
+    },
+    {
+      "epoch": 0.5571677307022634,
+      "grad_norm": 0.07144881784915924,
+      "learning_rate": 4.6821515892420544e-05,
+      "loss": 0.4462,
+      "step": 480
+    },
+    {
+      "epoch": 0.562971561230412,
+      "grad_norm": 0.052796777337789536,
+      "learning_rate": 4.6210268948655264e-05,
+      "loss": 0.4737,
+      "step": 485
+    },
+    {
+      "epoch": 0.5687753917585606,
+      "grad_norm": 0.05207886919379234,
+      "learning_rate": 4.559902200488998e-05,
+      "loss": 0.4773,
+      "step": 490
+    },
+    {
+      "epoch": 0.5745792222867092,
+      "grad_norm": 0.06733781844377518,
+      "learning_rate": 4.49877750611247e-05,
+      "loss": 0.4599,
+      "step": 495
+    },
+    {
+      "epoch": 0.5803830528148578,
+      "grad_norm": 0.05199211463332176,
+      "learning_rate": 4.437652811735942e-05,
+      "loss": 0.4673,
+      "step": 500
+    },
+    {
+      "epoch": 0.5861868833430064,
+      "grad_norm": 0.06036406010389328,
+      "learning_rate": 4.376528117359414e-05,
+      "loss": 0.474,
+      "step": 505
+    },
+    {
+      "epoch": 0.591990713871155,
+      "grad_norm": 0.05148075148463249,
+      "learning_rate": 4.315403422982885e-05,
+      "loss": 0.4776,
+      "step": 510
+    },
+    {
+      "epoch": 0.5977945443993036,
+      "grad_norm": 0.05437846854329109,
+      "learning_rate": 4.254278728606357e-05,
+      "loss": 0.4762,
+      "step": 515
+    },
+    {
+      "epoch": 0.6035983749274522,
+      "grad_norm": 0.06714485585689545,
+      "learning_rate": 4.193154034229829e-05,
+      "loss": 0.483,
+      "step": 520
+    },
+    {
+      "epoch": 0.6094022054556006,
+      "grad_norm": 0.05940714478492737,
+      "learning_rate": 4.132029339853301e-05,
+      "loss": 0.4732,
+      "step": 525
+    },
+    {
+      "epoch": 0.6152060359837492,
+      "grad_norm": 0.05213627591729164,
+      "learning_rate": 4.070904645476773e-05,
+      "loss": 0.4695,
+      "step": 530
+    },
+    {
+      "epoch": 0.6210098665118978,
+      "grad_norm": 0.05941450595855713,
+      "learning_rate": 4.0097799511002446e-05,
+      "loss": 0.4775,
+      "step": 535
+    },
+    {
+      "epoch": 0.6268136970400464,
+      "grad_norm": 0.06434749811887741,
+      "learning_rate": 3.9486552567237166e-05,
+      "loss": 0.4618,
+      "step": 540
+    },
+    {
+      "epoch": 0.632617527568195,
+      "grad_norm": 0.04946862533688545,
+      "learning_rate": 3.8875305623471886e-05,
+      "loss": 0.453,
+      "step": 545
+    },
+    {
+      "epoch": 0.6384213580963436,
+      "grad_norm": 0.0543401762843132,
+      "learning_rate": 3.8264058679706607e-05,
+      "loss": 0.4948,
+      "step": 550
+    },
+    {
+      "epoch": 0.6442251886244922,
+      "grad_norm": 0.05125496909022331,
+      "learning_rate": 3.765281173594133e-05,
+      "loss": 0.4578,
+      "step": 555
+    },
+    {
+      "epoch": 0.6500290191526408,
+      "grad_norm": 0.05509857088327408,
+      "learning_rate": 3.704156479217604e-05,
+      "loss": 0.4643,
+      "step": 560
+    },
+    {
+      "epoch": 0.6558328496807894,
+      "grad_norm": 0.046249207109212875,
+      "learning_rate": 3.643031784841076e-05,
+      "loss": 0.4793,
+      "step": 565
+    },
+    {
+      "epoch": 0.6616366802089378,
+      "grad_norm": 0.06396552920341492,
+      "learning_rate": 3.581907090464548e-05,
+      "loss": 0.4656,
+      "step": 570
+    },
+    {
+      "epoch": 0.6674405107370864,
+      "grad_norm": 0.05344633758068085,
+      "learning_rate": 3.52078239608802e-05,
+      "loss": 0.4709,
+      "step": 575
+    },
+    {
+      "epoch": 0.673244341265235,
+      "grad_norm": 0.05097387358546257,
+      "learning_rate": 3.4596577017114914e-05,
+      "loss": 0.4678,
+      "step": 580
+    },
+    {
+      "epoch": 0.6790481717933836,
+      "grad_norm": 0.08271091431379318,
+      "learning_rate": 3.3985330073349635e-05,
+      "loss": 0.4799,
+      "step": 585
+    },
+    {
+      "epoch": 0.6848520023215322,
+      "grad_norm": 0.057184312492609024,
+      "learning_rate": 3.3374083129584355e-05,
+      "loss": 0.4805,
+      "step": 590
+    },
+    {
+      "epoch": 0.6906558328496808,
+      "grad_norm": 0.0770120918750763,
+      "learning_rate": 3.2762836185819075e-05,
+      "loss": 0.4707,
+      "step": 595
+    },
+    {
+      "epoch": 0.6964596633778294,
+      "grad_norm": 0.052280493080616,
+      "learning_rate": 3.2151589242053795e-05,
+      "loss": 0.4838,
+      "step": 600
+    },
+    {
+      "epoch": 0.702263493905978,
+      "grad_norm": 0.05588831752538681,
+      "learning_rate": 3.154034229828851e-05,
+      "loss": 0.4572,
+      "step": 605
+    },
+    {
+      "epoch": 0.7080673244341266,
+      "grad_norm": 0.0569985955953598,
+      "learning_rate": 3.092909535452323e-05,
+      "loss": 0.4851,
+      "step": 610
+    },
+    {
+      "epoch": 0.7138711549622752,
+      "grad_norm": 0.06095046177506447,
+      "learning_rate": 3.031784841075795e-05,
+      "loss": 0.4751,
+      "step": 615
+    },
+    {
+      "epoch": 0.7196749854904236,
+      "grad_norm": 0.05518042668700218,
+      "learning_rate": 2.9706601466992666e-05,
+      "loss": 0.4605,
+      "step": 620
+    },
+    {
+      "epoch": 0.7254788160185722,
+      "grad_norm": 0.05149676650762558,
+      "learning_rate": 2.9095354523227386e-05,
+      "loss": 0.4486,
+      "step": 625
+    },
+    {
+      "epoch": 0.7312826465467208,
+      "grad_norm": 0.0650547444820404,
+      "learning_rate": 2.8484107579462103e-05,
+      "loss": 0.4678,
+      "step": 630
+    },
+    {
+      "epoch": 0.7370864770748694,
+      "grad_norm": 0.05262364447116852,
+      "learning_rate": 2.7872860635696824e-05,
+      "loss": 0.4625,
+      "step": 635
+    },
+    {
+      "epoch": 0.742890307603018,
+      "grad_norm": 0.04777299985289574,
+      "learning_rate": 2.7261613691931544e-05,
+      "loss": 0.4816,
+      "step": 640
+    },
+    {
+      "epoch": 0.7486941381311666,
+      "grad_norm": 0.044535353779792786,
+      "learning_rate": 2.665036674816626e-05,
+      "loss": 0.4924,
+      "step": 645
+    },
+    {
+      "epoch": 0.7544979686593152,
+      "grad_norm": 0.04823042452335358,
+      "learning_rate": 2.603911980440098e-05,
+      "loss": 0.4725,
+      "step": 650
+    },
+    {
+      "epoch": 0.7603017991874638,
+      "grad_norm": 0.057258885353803635,
+      "learning_rate": 2.5427872860635698e-05,
+      "loss": 0.454,
+      "step": 655
+    },
+    {
+      "epoch": 0.7661056297156124,
+      "grad_norm": 0.05706983432173729,
+      "learning_rate": 2.4816625916870418e-05,
+      "loss": 0.4628,
+      "step": 660
+    },
+    {
+      "epoch": 0.7719094602437608,
+      "grad_norm": 0.0592195987701416,
+      "learning_rate": 2.4205378973105135e-05,
+      "loss": 0.55,
+      "step": 665
+    },
+    {
+      "epoch": 0.7777132907719094,
+      "grad_norm": 0.05927932262420654,
+      "learning_rate": 2.3594132029339855e-05,
+      "loss": 0.4508,
+      "step": 670
+    },
+    {
+      "epoch": 0.783517121300058,
+      "grad_norm": 0.05543314665555954,
+      "learning_rate": 2.2982885085574575e-05,
+      "loss": 0.4405,
+      "step": 675
+    },
+    {
+      "epoch": 0.7893209518282066,
+      "grad_norm": 0.05768333375453949,
+      "learning_rate": 2.2371638141809292e-05,
+      "loss": 0.4656,
+      "step": 680
+    },
+    {
+      "epoch": 0.7951247823563552,
+      "grad_norm": 0.05134601518511772,
+      "learning_rate": 2.1760391198044012e-05,
+      "loss": 0.4842,
+      "step": 685
+    },
+    {
+      "epoch": 0.8009286128845038,
+      "grad_norm": 0.05869955196976662,
+      "learning_rate": 2.114914425427873e-05,
+      "loss": 0.4586,
+      "step": 690
+    },
+    {
+      "epoch": 0.8067324434126524,
+      "grad_norm": 0.06825531274080276,
+      "learning_rate": 2.053789731051345e-05,
+      "loss": 0.483,
+      "step": 695
+    },
+    {
+      "epoch": 0.812536273940801,
+      "grad_norm": 0.0546259880065918,
+      "learning_rate": 1.9926650366748166e-05,
+      "loss": 0.4853,
+      "step": 700
+    },
+    {
+      "epoch": 0.8183401044689496,
+      "grad_norm": 0.05420086532831192,
+      "learning_rate": 1.9315403422982887e-05,
+      "loss": 0.4591,
+      "step": 705
+    },
+    {
+      "epoch": 0.824143934997098,
+      "grad_norm": 0.053969256579875946,
+      "learning_rate": 1.8704156479217603e-05,
+      "loss": 0.4618,
+      "step": 710
+    },
+    {
+      "epoch": 0.8299477655252466,
+      "grad_norm": 0.050985366106033325,
+      "learning_rate": 1.8092909535452324e-05,
+      "loss": 0.4769,
+      "step": 715
+    },
+    {
+      "epoch": 0.8357515960533952,
+      "grad_norm": 0.060406915843486786,
+      "learning_rate": 1.7481662591687044e-05,
+      "loss": 0.4872,
+      "step": 720
+    },
+    {
+      "epoch": 0.8415554265815438,
+      "grad_norm": 0.06030416861176491,
+      "learning_rate": 1.687041564792176e-05,
+      "loss": 0.4692,
+      "step": 725
+    },
+    {
+      "epoch": 0.8473592571096924,
+      "grad_norm": 0.050093889236450195,
+      "learning_rate": 1.625916870415648e-05,
+      "loss": 0.4622,
+      "step": 730
+    },
+    {
+      "epoch": 0.853163087637841,
+      "grad_norm": 0.05956275388598442,
+      "learning_rate": 1.5647921760391198e-05,
+      "loss": 0.4923,
+      "step": 735
+    },
+    {
+      "epoch": 0.8589669181659896,
+      "grad_norm": 0.04703729227185249,
+      "learning_rate": 1.5036674816625918e-05,
+      "loss": 0.4736,
+      "step": 740
+    },
+    {
+      "epoch": 0.8647707486941382,
+      "grad_norm": 0.0508451983332634,
+      "learning_rate": 1.4425427872860637e-05,
+      "loss": 0.4606,
+      "step": 745
+    },
+    {
+      "epoch": 0.8705745792222868,
+      "grad_norm": 0.050170641392469406,
+      "learning_rate": 1.3814180929095355e-05,
+      "loss": 0.4641,
+      "step": 750
+    },
+    {
+      "epoch": 0.8763784097504352,
+      "grad_norm": 0.06078166142106056,
+      "learning_rate": 1.3202933985330074e-05,
+      "loss": 0.4389,
+      "step": 755
+    },
+    {
+      "epoch": 0.8821822402785838,
+      "grad_norm": 0.05649907514452934,
+      "learning_rate": 1.2591687041564792e-05,
+      "loss": 0.4687,
+      "step": 760
+    },
+    {
+      "epoch": 0.8879860708067324,
+      "grad_norm": 0.050264623016119,
+      "learning_rate": 1.198044009779951e-05,
+      "loss": 0.5273,
+      "step": 765
+    },
+    {
+      "epoch": 0.893789901334881,
+      "grad_norm": 0.058568935841321945,
+      "learning_rate": 1.136919315403423e-05,
+      "loss": 0.4628,
+      "step": 770
+    },
+    {
+      "epoch": 0.8995937318630296,
+      "grad_norm": 0.06023811176419258,
+      "learning_rate": 1.075794621026895e-05,
+      "loss": 0.4648,
+      "step": 775
+    },
+    {
+      "epoch": 0.9053975623911782,
+      "grad_norm": 0.06913120299577713,
+      "learning_rate": 1.0146699266503668e-05,
+      "loss": 0.4527,
+      "step": 780
+    },
+    {
+      "epoch": 0.9112013929193268,
+      "grad_norm": 0.05033561587333679,
+      "learning_rate": 9.535452322738387e-06,
+      "loss": 0.4733,
+      "step": 785
+    },
+    {
+      "epoch": 0.9170052234474754,
+      "grad_norm": 0.048522599041461945,
+      "learning_rate": 8.924205378973105e-06,
+      "loss": 0.4608,
+      "step": 790
+    },
+    {
+      "epoch": 0.922809053975624,
+      "grad_norm": 0.055790580809116364,
+      "learning_rate": 8.312958435207824e-06,
+      "loss": 0.4718,
+      "step": 795
+    },
+    {
+      "epoch": 0.9286128845037724,
+      "grad_norm": 0.05398460850119591,
+      "learning_rate": 7.701711491442542e-06,
+      "loss": 0.4586,
+      "step": 800
+    },
+    {
+      "epoch": 0.934416715031921,
+      "grad_norm": 0.06667961925268173,
+      "learning_rate": 7.090464547677262e-06,
+      "loss": 0.4301,
+      "step": 805
+    },
+    {
+      "epoch": 0.9402205455600696,
+      "grad_norm": 0.0602191723883152,
+      "learning_rate": 6.47921760391198e-06,
+      "loss": 0.4521,
+      "step": 810
+    },
+    {
+      "epoch": 0.9460243760882182,
+      "grad_norm": 0.0520971417427063,
+      "learning_rate": 5.8679706601467e-06,
+      "loss": 0.4933,
+      "step": 815
+    },
+    {
+      "epoch": 0.9518282066163668,
+      "grad_norm": 0.06370063871145248,
+      "learning_rate": 5.256723716381418e-06,
+      "loss": 0.4627,
+      "step": 820
+    },
+    {
+      "epoch": 0.9576320371445154,
+      "grad_norm": 0.059502165764570236,
+      "learning_rate": 4.645476772616137e-06,
+      "loss": 0.4674,
+      "step": 825
+    },
+    {
+      "epoch": 0.963435867672664,
+      "grad_norm": 0.04978031665086746,
+      "learning_rate": 4.034229828850856e-06,
+      "loss": 0.4419,
+      "step": 830
+    },
+    {
+      "epoch": 0.9692396982008126,
+      "grad_norm": 0.055051643401384354,
+      "learning_rate": 3.4229828850855746e-06,
+      "loss": 0.4724,
+      "step": 835
+    },
+    {
+      "epoch": 0.9750435287289612,
+      "grad_norm": 0.15939809381961823,
+      "learning_rate": 2.8117359413202936e-06,
+      "loss": 0.4677,
+      "step": 840
+    },
+    {
+      "epoch": 0.9808473592571096,
+      "grad_norm": 0.05436515808105469,
+      "learning_rate": 2.2004889975550126e-06,
+      "loss": 0.4723,
+      "step": 845
+    },
+    {
+      "epoch": 0.9866511897852582,
+      "grad_norm": 0.052108634263277054,
+      "learning_rate": 1.5892420537897311e-06,
+      "loss": 0.4798,
+      "step": 850
+    },
+    {
+      "epoch": 0.9924550203134068,
+      "grad_norm": 0.08515449613332748,
+      "learning_rate": 9.779951100244499e-07,
+      "loss": 0.4344,
+      "step": 855
+    },
+    {
+      "epoch": 0.9982588508415554,
+      "grad_norm": 0.057271551340818405,
+      "learning_rate": 3.667481662591687e-07,
+      "loss": 0.4639,
+      "step": 860
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 862,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.192258616665068e+20,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Qwen3-30B-A3B-Thinking-2507-sft-fusang-swa=2k_sink=100_falayer=25_from0_step2_fadec/checkpoint-862/README.md

@@ -0,0 +1,207 @@
+---
+base_model: /share/models/Qwen3-30B-A3B-Thinking-2507
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/share/models/Qwen3-30B-A3B-Thinking-2507
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1

Qwen3-30B-A3B-Thinking-2507-sft-fusang-swa=2k_sink=100_falayer=25_from0_step2_fadec/checkpoint-862/adapter_config.json

@@ -0,0 +1,38 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/share/models/Qwen3-30B-A3B-Thinking-2507",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj",
+    "k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": true
+}

Qwen3-30B-A3B-Thinking-2507-sft-fusang-swa=2k_sink=100_falayer=25_from0_step2_fadec/checkpoint-862/chat_template.jinja

@@ -0,0 +1,86 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
+        {%- set ns.multi_step_tool = false %}
+        {%- set ns.last_query_index = index %}
+    {%- endif %}
+{%- endfor %}
+{%- for message in messages %}
+    {%- if message.content is string %}
+        {%- set content = message.content %}
+    {%- else %}
+        {%- set content = '' %}
+    {%- endif %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {%- if loop.last or (not loop.last and reasoning_content) %}
+                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
+            {%- else %}
+                {{- '<|im_start|>' + message.role + '\n' + content }}
+            {%- endif %}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n<think>\n' }}
+{%- endif %}

Qwen3-30B-A3B-Thinking-2507-sft-fusang-swa=2k_sink=100_falayer=25_from0_step2_fadec/checkpoint-862/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Qwen3-30B-A3B-Thinking-2507-sft-fusang-swa=2k_sink=100_falayer=25_from0_step2_fadec/checkpoint-862/tokenizer_config.json

@@ -0,0 +1,239 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151665": {
+      "content": "<tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151666": {
+      "content": "</tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151667": {
+      "content": "<think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151668": {
+      "content": "</think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 1010000,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

Qwen3-30B-A3B-Thinking-2507-sft-fusang-swa=2k_sink=100_falayer=25_from0_step2_fadec/checkpoint-862/trainer_state.json

@@ -0,0 +1,1238 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.0,
+  "eval_steps": 500,
+  "global_step": 862,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.005803830528148578,
+      "grad_norm": 0.052840907126665115,
+      "learning_rate": 9.090909090909091e-06,
+      "loss": 0.4226,
+      "step": 5
+    },
+    {
+      "epoch": 0.011607661056297156,
+      "grad_norm": 0.054803915321826935,
+      "learning_rate": 2.0454545454545457e-05,
+      "loss": 0.4498,
+      "step": 10
+    },
+    {
+      "epoch": 0.017411491584445733,
+      "grad_norm": 0.04625187814235687,
+      "learning_rate": 3.181818181818182e-05,
+      "loss": 0.4624,
+      "step": 15
+    },
+    {
+      "epoch": 0.023215322112594312,
+      "grad_norm": 0.049974825233221054,
+      "learning_rate": 4.318181818181819e-05,
+      "loss": 0.4548,
+      "step": 20
+    },
+    {
+      "epoch": 0.02901915264074289,
+      "grad_norm": 0.055535830557346344,
+      "learning_rate": 5.4545454545454546e-05,
+      "loss": 0.4495,
+      "step": 25
+    },
+    {
+      "epoch": 0.034822983168891465,
+      "grad_norm": 0.05944204702973366,
+      "learning_rate": 6.59090909090909e-05,
+      "loss": 0.4342,
+      "step": 30
+    },
+    {
+      "epoch": 0.04062681369704005,
+      "grad_norm": 0.045468952506780624,
+      "learning_rate": 7.727272727272727e-05,
+      "loss": 0.4411,
+      "step": 35
+    },
+    {
+      "epoch": 0.046430644225188625,
+      "grad_norm": 0.053401365876197815,
+      "learning_rate": 8.863636363636364e-05,
+      "loss": 0.4534,
+      "step": 40
+    },
+    {
+      "epoch": 0.0522344747533372,
+      "grad_norm": 0.03725573047995567,
+      "learning_rate": 0.0001,
+      "loss": 0.4543,
+      "step": 45
+    },
+    {
+      "epoch": 0.05803830528148578,
+      "grad_norm": 0.051740411669015884,
+      "learning_rate": 9.938875305623473e-05,
+      "loss": 0.4613,
+      "step": 50
+    },
+    {
+      "epoch": 0.06384213580963435,
+      "grad_norm": 0.051915571093559265,
+      "learning_rate": 9.877750611246945e-05,
+      "loss": 0.454,
+      "step": 55
+    },
+    {
+      "epoch": 0.06964596633778293,
+      "grad_norm": 0.06379981338977814,
+      "learning_rate": 9.816625916870417e-05,
+      "loss": 0.4537,
+      "step": 60
+    },
+    {
+      "epoch": 0.07544979686593152,
+      "grad_norm": 0.04472048208117485,
+      "learning_rate": 9.755501222493889e-05,
+      "loss": 0.4432,
+      "step": 65
+    },
+    {
+      "epoch": 0.0812536273940801,
+      "grad_norm": 0.039609394967556,
+      "learning_rate": 9.69437652811736e-05,
+      "loss": 0.44,
+      "step": 70
+    },
+    {
+      "epoch": 0.08705745792222867,
+      "grad_norm": 0.046134259551763535,
+      "learning_rate": 9.633251833740831e-05,
+      "loss": 0.479,
+      "step": 75
+    },
+    {
+      "epoch": 0.09286128845037725,
+      "grad_norm": 0.04198378324508667,
+      "learning_rate": 9.572127139364305e-05,
+      "loss": 0.4296,
+      "step": 80
+    },
+    {
+      "epoch": 0.09866511897852583,
+      "grad_norm": 0.04254145547747612,
+      "learning_rate": 9.511002444987775e-05,
+      "loss": 0.4358,
+      "step": 85
+    },
+    {
+      "epoch": 0.1044689495066744,
+      "grad_norm": 0.0387021042406559,
+      "learning_rate": 9.449877750611247e-05,
+      "loss": 0.4416,
+      "step": 90
+    },
+    {
+      "epoch": 0.11027278003482298,
+      "grad_norm": 0.03391820192337036,
+      "learning_rate": 9.38875305623472e-05,
+      "loss": 0.4402,
+      "step": 95
+    },
+    {
+      "epoch": 0.11607661056297155,
+      "grad_norm": 0.04681016504764557,
+      "learning_rate": 9.327628361858191e-05,
+      "loss": 0.4555,
+      "step": 100
+    },
+    {
+      "epoch": 0.12188044109112015,
+      "grad_norm": 0.043560221791267395,
+      "learning_rate": 9.266503667481663e-05,
+      "loss": 0.4388,
+      "step": 105
+    },
+    {
+      "epoch": 0.1276842716192687,
+      "grad_norm": 0.04997171461582184,
+      "learning_rate": 9.205378973105135e-05,
+      "loss": 0.4408,
+      "step": 110
+    },
+    {
+      "epoch": 0.1334881021474173,
+      "grad_norm": 0.03899248316884041,
+      "learning_rate": 9.144254278728606e-05,
+      "loss": 0.4543,
+      "step": 115
+    },
+    {
+      "epoch": 0.13929193267556586,
+      "grad_norm": 0.03691416233778,
+      "learning_rate": 9.08312958435208e-05,
+      "loss": 0.459,
+      "step": 120
+    },
+    {
+      "epoch": 0.14509576320371445,
+      "grad_norm": 0.045570001006126404,
+      "learning_rate": 9.02200488997555e-05,
+      "loss": 0.4354,
+      "step": 125
+    },
+    {
+      "epoch": 0.15089959373186304,
+      "grad_norm": 0.040693432092666626,
+      "learning_rate": 8.960880195599024e-05,
+      "loss": 0.4484,
+      "step": 130
+    },
+    {
+      "epoch": 0.1567034242600116,
+      "grad_norm": 0.03382732346653938,
+      "learning_rate": 8.899755501222494e-05,
+      "loss": 0.4571,
+      "step": 135
+    },
+    {
+      "epoch": 0.1625072547881602,
+      "grad_norm": 0.03577801212668419,
+      "learning_rate": 8.838630806845966e-05,
+      "loss": 0.4689,
+      "step": 140
+    },
+    {
+      "epoch": 0.16831108531630876,
+      "grad_norm": 0.040085580199956894,
+      "learning_rate": 8.777506112469438e-05,
+      "loss": 0.4626,
+      "step": 145
+    },
+    {
+      "epoch": 0.17411491584445735,
+      "grad_norm": 0.04042517766356468,
+      "learning_rate": 8.71638141809291e-05,
+      "loss": 0.4527,
+      "step": 150
+    },
+    {
+      "epoch": 0.1799187463726059,
+      "grad_norm": 0.03714785352349281,
+      "learning_rate": 8.655256723716382e-05,
+      "loss": 0.4274,
+      "step": 155
+    },
+    {
+      "epoch": 0.1857225769007545,
+      "grad_norm": 0.03365296497941017,
+      "learning_rate": 8.594132029339854e-05,
+      "loss": 0.4488,
+      "step": 160
+    },
+    {
+      "epoch": 0.1915264074289031,
+      "grad_norm": 0.040339600294828415,
+      "learning_rate": 8.533007334963325e-05,
+      "loss": 0.4491,
+      "step": 165
+    },
+    {
+      "epoch": 0.19733023795705165,
+      "grad_norm": 0.030923722311854362,
+      "learning_rate": 8.471882640586798e-05,
+      "loss": 0.4486,
+      "step": 170
+    },
+    {
+      "epoch": 0.20313406848520024,
+      "grad_norm": 0.04013897478580475,
+      "learning_rate": 8.410757946210269e-05,
+      "loss": 0.4545,
+      "step": 175
+    },
+    {
+      "epoch": 0.2089378990133488,
+      "grad_norm": 0.03853878378868103,
+      "learning_rate": 8.349633251833741e-05,
+      "loss": 0.4532,
+      "step": 180
+    },
+    {
+      "epoch": 0.2147417295414974,
+      "grad_norm": 0.04753940552473068,
+      "learning_rate": 8.288508557457213e-05,
+      "loss": 0.4539,
+      "step": 185
+    },
+    {
+      "epoch": 0.22054556006964596,
+      "grad_norm": 0.02661183848977089,
+      "learning_rate": 8.227383863080685e-05,
+      "loss": 0.4484,
+      "step": 190
+    },
+    {
+      "epoch": 0.22634939059779455,
+      "grad_norm": 0.04699932783842087,
+      "learning_rate": 8.166259168704157e-05,
+      "loss": 0.4257,
+      "step": 195
+    },
+    {
+      "epoch": 0.2321532211259431,
+      "grad_norm": 0.03680172935128212,
+      "learning_rate": 8.105134474327629e-05,
+      "loss": 0.4443,
+      "step": 200
+    },
+    {
+      "epoch": 0.2379570516540917,
+      "grad_norm": 0.0349036268889904,
+      "learning_rate": 8.044009779951101e-05,
+      "loss": 0.4385,
+      "step": 205
+    },
+    {
+      "epoch": 0.2437608821822403,
+      "grad_norm": 0.03773039951920509,
+      "learning_rate": 7.982885085574573e-05,
+      "loss": 0.4725,
+      "step": 210
+    },
+    {
+      "epoch": 0.24956471271038885,
+      "grad_norm": 0.03726497292518616,
+      "learning_rate": 7.921760391198044e-05,
+      "loss": 0.4454,
+      "step": 215
+    },
+    {
+      "epoch": 0.2553685432385374,
+      "grad_norm": 0.026751970872282982,
+      "learning_rate": 7.860635696821517e-05,
+      "loss": 0.432,
+      "step": 220
+    },
+    {
+      "epoch": 0.261172373766686,
+      "grad_norm": 0.03596620634198189,
+      "learning_rate": 7.799511002444988e-05,
+      "loss": 0.4538,
+      "step": 225
+    },
+    {
+      "epoch": 0.2669762042948346,
+      "grad_norm": 0.045810550451278687,
+      "learning_rate": 7.73838630806846e-05,
+      "loss": 0.4391,
+      "step": 230
+    },
+    {
+      "epoch": 0.2727800348229832,
+      "grad_norm": 0.031115932390093803,
+      "learning_rate": 7.677261613691932e-05,
+      "loss": 0.4669,
+      "step": 235
+    },
+    {
+      "epoch": 0.2785838653511317,
+      "grad_norm": 0.030680067837238312,
+      "learning_rate": 7.616136919315404e-05,
+      "loss": 0.4543,
+      "step": 240
+    },
+    {
+      "epoch": 0.2843876958792803,
+      "grad_norm": 0.027802886441349983,
+      "learning_rate": 7.555012224938876e-05,
+      "loss": 0.4432,
+      "step": 245
+    },
+    {
+      "epoch": 0.2901915264074289,
+      "grad_norm": 0.03515085205435753,
+      "learning_rate": 7.493887530562348e-05,
+      "loss": 0.4669,
+      "step": 250
+    },
+    {
+      "epoch": 0.2959953569355775,
+      "grad_norm": 0.056282296776771545,
+      "learning_rate": 7.432762836185819e-05,
+      "loss": 0.4456,
+      "step": 255
+    },
+    {
+      "epoch": 0.3017991874637261,
+      "grad_norm": 0.03418450802564621,
+      "learning_rate": 7.371638141809292e-05,
+      "loss": 0.4683,
+      "step": 260
+    },
+    {
+      "epoch": 0.3076030179918746,
+      "grad_norm": 0.03879056125879288,
+      "learning_rate": 7.310513447432763e-05,
+      "loss": 0.4339,
+      "step": 265
+    },
+    {
+      "epoch": 0.3134068485200232,
+      "grad_norm": 0.027370406314730644,
+      "learning_rate": 7.249388753056235e-05,
+      "loss": 0.4383,
+      "step": 270
+    },
+    {
+      "epoch": 0.3192106790481718,
+      "grad_norm": 0.03739843890070915,
+      "learning_rate": 7.188264058679707e-05,
+      "loss": 0.4528,
+      "step": 275
+    },
+    {
+      "epoch": 0.3250145095763204,
+      "grad_norm": 1.0982234477996826,
+      "learning_rate": 7.127139364303179e-05,
+      "loss": 0.4551,
+      "step": 280
+    },
+    {
+      "epoch": 0.3308183401044689,
+      "grad_norm": 0.03400462120771408,
+      "learning_rate": 7.066014669926651e-05,
+      "loss": 0.4675,
+      "step": 285
+    },
+    {
+      "epoch": 0.3366221706326175,
+      "grad_norm": 0.042476948350667953,
+      "learning_rate": 7.004889975550123e-05,
+      "loss": 0.449,
+      "step": 290
+    },
+    {
+      "epoch": 0.3424260011607661,
+      "grad_norm": 0.029475195333361626,
+      "learning_rate": 6.943765281173595e-05,
+      "loss": 0.4486,
+      "step": 295
+    },
+    {
+      "epoch": 0.3482298316889147,
+      "grad_norm": 0.05517793074250221,
+      "learning_rate": 6.882640586797067e-05,
+      "loss": 0.4516,
+      "step": 300
+    },
+    {
+      "epoch": 0.3540336622170633,
+      "grad_norm": 0.036910708993673325,
+      "learning_rate": 6.821515892420538e-05,
+      "loss": 0.4339,
+      "step": 305
+    },
+    {
+      "epoch": 0.3598374927452118,
+      "grad_norm": 0.0337318480014801,
+      "learning_rate": 6.760391198044011e-05,
+      "loss": 0.4657,
+      "step": 310
+    },
+    {
+      "epoch": 0.3656413232733604,
+      "grad_norm": 0.03524719178676605,
+      "learning_rate": 6.699266503667482e-05,
+      "loss": 0.4465,
+      "step": 315
+    },
+    {
+      "epoch": 0.371445153801509,
+      "grad_norm": 0.027205439284443855,
+      "learning_rate": 6.638141809290954e-05,
+      "loss": 0.4384,
+      "step": 320
+    },
+    {
+      "epoch": 0.3772489843296576,
+      "grad_norm": 0.03811550512909889,
+      "learning_rate": 6.577017114914426e-05,
+      "loss": 0.4641,
+      "step": 325
+    },
+    {
+      "epoch": 0.3830528148578062,
+      "grad_norm": 0.03354226052761078,
+      "learning_rate": 6.515892420537898e-05,
+      "loss": 0.4377,
+      "step": 330
+    },
+    {
+      "epoch": 0.3888566453859547,
+      "grad_norm": 0.03483116254210472,
+      "learning_rate": 6.45476772616137e-05,
+      "loss": 0.4516,
+      "step": 335
+    },
+    {
+      "epoch": 0.3946604759141033,
+      "grad_norm": 0.030540024861693382,
+      "learning_rate": 6.393643031784842e-05,
+      "loss": 0.4424,
+      "step": 340
+    },
+    {
+      "epoch": 0.4004643064422519,
+      "grad_norm": 0.0276072658598423,
+      "learning_rate": 6.332518337408312e-05,
+      "loss": 0.4531,
+      "step": 345
+    },
+    {
+      "epoch": 0.4062681369704005,
+      "grad_norm": 0.03627203777432442,
+      "learning_rate": 6.271393643031786e-05,
+      "loss": 0.433,
+      "step": 350
+    },
+    {
+      "epoch": 0.412071967498549,
+      "grad_norm": 0.037168748676776886,
+      "learning_rate": 6.210268948655256e-05,
+      "loss": 0.444,
+      "step": 355
+    },
+    {
+      "epoch": 0.4178757980266976,
+      "grad_norm": 0.03232443332672119,
+      "learning_rate": 6.14914425427873e-05,
+      "loss": 0.4685,
+      "step": 360
+    },
+    {
+      "epoch": 0.4236796285548462,
+      "grad_norm": 0.0315212681889534,
+      "learning_rate": 6.0880195599022005e-05,
+      "loss": 0.4416,
+      "step": 365
+    },
+    {
+      "epoch": 0.4294834590829948,
+      "grad_norm": 0.03185131400823593,
+      "learning_rate": 6.026894865525673e-05,
+      "loss": 0.4369,
+      "step": 370
+    },
+    {
+      "epoch": 0.4352872896111434,
+      "grad_norm": 0.037931546568870544,
+      "learning_rate": 5.9657701711491446e-05,
+      "loss": 0.4652,
+      "step": 375
+    },
+    {
+      "epoch": 0.4410911201392919,
+      "grad_norm": 0.03081417828798294,
+      "learning_rate": 5.9046454767726166e-05,
+      "loss": 0.4341,
+      "step": 380
+    },
+    {
+      "epoch": 0.4468949506674405,
+      "grad_norm": 0.04330901429057121,
+      "learning_rate": 5.843520782396088e-05,
+      "loss": 0.4251,
+      "step": 385
+    },
+    {
+      "epoch": 0.4526987811955891,
+      "grad_norm": 0.028708400204777718,
+      "learning_rate": 5.7823960880195606e-05,
+      "loss": 0.4508,
+      "step": 390
+    },
+    {
+      "epoch": 0.4585026117237377,
+      "grad_norm": 0.02889464609324932,
+      "learning_rate": 5.721271393643032e-05,
+      "loss": 0.4361,
+      "step": 395
+    },
+    {
+      "epoch": 0.4643064422518862,
+      "grad_norm": 0.038650140166282654,
+      "learning_rate": 5.660146699266504e-05,
+      "loss": 0.4455,
+      "step": 400
+    },
+    {
+      "epoch": 0.4701102727800348,
+      "grad_norm": 0.034687239676713943,
+      "learning_rate": 5.5990220048899754e-05,
+      "loss": 0.4421,
+      "step": 405
+    },
+    {
+      "epoch": 0.4759141033081834,
+      "grad_norm": 0.028834836557507515,
+      "learning_rate": 5.537897310513448e-05,
+      "loss": 0.4293,
+      "step": 410
+    },
+    {
+      "epoch": 0.481717933836332,
+      "grad_norm": 0.029916927218437195,
+      "learning_rate": 5.4767726161369194e-05,
+      "loss": 0.4498,
+      "step": 415
+    },
+    {
+      "epoch": 0.4875217643644806,
+      "grad_norm": 0.029675010591745377,
+      "learning_rate": 5.4156479217603914e-05,
+      "loss": 0.4408,
+      "step": 420
+    },
+    {
+      "epoch": 0.4933255948926291,
+      "grad_norm": 0.038431085646152496,
+      "learning_rate": 5.354523227383863e-05,
+      "loss": 0.4485,
+      "step": 425
+    },
+    {
+      "epoch": 0.4991294254207777,
+      "grad_norm": 0.03521041199564934,
+      "learning_rate": 5.2933985330073355e-05,
+      "loss": 0.4362,
+      "step": 430
+    },
+    {
+      "epoch": 0.5049332559489262,
+      "grad_norm": 0.029747577384114265,
+      "learning_rate": 5.232273838630807e-05,
+      "loss": 0.4607,
+      "step": 435
+    },
+    {
+      "epoch": 0.5107370864770748,
+      "grad_norm": 0.0325702428817749,
+      "learning_rate": 5.1711491442542795e-05,
+      "loss": 0.4575,
+      "step": 440
+    },
+    {
+      "epoch": 0.5165409170052234,
+      "grad_norm": 0.03319084644317627,
+      "learning_rate": 5.110024449877751e-05,
+      "loss": 0.4581,
+      "step": 445
+    },
+    {
+      "epoch": 0.522344747533372,
+      "grad_norm": 0.03660687804222107,
+      "learning_rate": 5.048899755501223e-05,
+      "loss": 0.4685,
+      "step": 450
+    },
+    {
+      "epoch": 0.5281485780615206,
+      "grad_norm": 0.033311598002910614,
+      "learning_rate": 4.987775061124695e-05,
+      "loss": 0.4358,
+      "step": 455
+    },
+    {
+      "epoch": 0.5339524085896692,
+      "grad_norm": 0.028579257428646088,
+      "learning_rate": 4.926650366748167e-05,
+      "loss": 0.4536,
+      "step": 460
+    },
+    {
+      "epoch": 0.5397562391178178,
+      "grad_norm": 0.028189752250909805,
+      "learning_rate": 4.865525672371638e-05,
+      "loss": 0.4578,
+      "step": 465
+    },
+    {
+      "epoch": 0.5455600696459664,
+      "grad_norm": 0.03154165670275688,
+      "learning_rate": 4.80440097799511e-05,
+      "loss": 0.4569,
+      "step": 470
+    },
+    {
+      "epoch": 0.551363900174115,
+      "grad_norm": 0.04257906228303909,
+      "learning_rate": 4.743276283618582e-05,
+      "loss": 0.4343,
+      "step": 475
+    },
+    {
+      "epoch": 0.5571677307022634,
+      "grad_norm": 0.03221024572849274,
+      "learning_rate": 4.6821515892420544e-05,
+      "loss": 0.4216,
+      "step": 480
+    },
+    {
+      "epoch": 0.562971561230412,
+      "grad_norm": 0.030446268618106842,
+      "learning_rate": 4.6210268948655264e-05,
+      "loss": 0.4515,
+      "step": 485
+    },
+    {
+      "epoch": 0.5687753917585606,
+      "grad_norm": 0.028062935918569565,
+      "learning_rate": 4.559902200488998e-05,
+      "loss": 0.4557,
+      "step": 490
+    },
+    {
+      "epoch": 0.5745792222867092,
+      "grad_norm": 0.0328240841627121,
+      "learning_rate": 4.49877750611247e-05,
+      "loss": 0.4353,
+      "step": 495
+    },
+    {
+      "epoch": 0.5803830528148578,
+      "grad_norm": 0.033599432557821274,
+      "learning_rate": 4.437652811735942e-05,
+      "loss": 0.445,
+      "step": 500
+    },
+    {
+      "epoch": 0.5861868833430064,
+      "grad_norm": 0.03244630992412567,
+      "learning_rate": 4.376528117359414e-05,
+      "loss": 0.4491,
+      "step": 505
+    },
+    {
+      "epoch": 0.591990713871155,
+      "grad_norm": 0.03058663383126259,
+      "learning_rate": 4.315403422982885e-05,
+      "loss": 0.4543,
+      "step": 510
+    },
+    {
+      "epoch": 0.5977945443993036,
+      "grad_norm": 0.027349628508090973,
+      "learning_rate": 4.254278728606357e-05,
+      "loss": 0.4532,
+      "step": 515
+    },
+    {
+      "epoch": 0.6035983749274522,
+      "grad_norm": 0.028902901336550713,
+      "learning_rate": 4.193154034229829e-05,
+      "loss": 0.4592,
+      "step": 520
+    },
+    {
+      "epoch": 0.6094022054556006,
+      "grad_norm": 0.037853822112083435,
+      "learning_rate": 4.132029339853301e-05,
+      "loss": 0.4509,
+      "step": 525
+    },
+    {
+      "epoch": 0.6152060359837492,
+      "grad_norm": 0.03172282129526138,
+      "learning_rate": 4.070904645476773e-05,
+      "loss": 0.4458,
+      "step": 530
+    },
+    {
+      "epoch": 0.6210098665118978,
+      "grad_norm": 0.032361604273319244,
+      "learning_rate": 4.0097799511002446e-05,
+      "loss": 0.4539,
+      "step": 535
+    },
+    {
+      "epoch": 0.6268136970400464,
+      "grad_norm": 0.029296280816197395,
+      "learning_rate": 3.9486552567237166e-05,
+      "loss": 0.4391,
+      "step": 540
+    },
+    {
+      "epoch": 0.632617527568195,
+      "grad_norm": 0.031962957233190536,
+      "learning_rate": 3.8875305623471886e-05,
+      "loss": 0.4304,
+      "step": 545
+    },
+    {
+      "epoch": 0.6384213580963436,
+      "grad_norm": 0.03842572122812271,
+      "learning_rate": 3.8264058679706607e-05,
+      "loss": 0.4718,
+      "step": 550
+    },
+    {
+      "epoch": 0.6442251886244922,
+      "grad_norm": 0.03261444717645645,
+      "learning_rate": 3.765281173594133e-05,
+      "loss": 0.4351,
+      "step": 555
+    },
+    {
+      "epoch": 0.6500290191526408,
+      "grad_norm": 0.03987475857138634,
+      "learning_rate": 3.704156479217604e-05,
+      "loss": 0.4425,
+      "step": 560
+    },
+    {
+      "epoch": 0.6558328496807894,
+      "grad_norm": 0.026505326852202415,
+      "learning_rate": 3.643031784841076e-05,
+      "loss": 0.4578,
+      "step": 565
+    },
+    {
+      "epoch": 0.6616366802089378,
+      "grad_norm": 0.03378090634942055,
+      "learning_rate": 3.581907090464548e-05,
+      "loss": 0.4434,
+      "step": 570
+    },
+    {
+      "epoch": 0.6674405107370864,
+      "grad_norm": 0.03907475247979164,
+      "learning_rate": 3.52078239608802e-05,
+      "loss": 0.4499,
+      "step": 575
+    },
+    {
+      "epoch": 0.673244341265235,
+      "grad_norm": 0.03311920911073685,
+      "learning_rate": 3.4596577017114914e-05,
+      "loss": 0.4446,
+      "step": 580
+    },
+    {
+      "epoch": 0.6790481717933836,
+      "grad_norm": 0.031724512577056885,
+      "learning_rate": 3.3985330073349635e-05,
+      "loss": 0.4579,
+      "step": 585
+    },
+    {
+      "epoch": 0.6848520023215322,
+      "grad_norm": 0.029358413070440292,
+      "learning_rate": 3.3374083129584355e-05,
+      "loss": 0.4578,
+      "step": 590
+    },
+    {
+      "epoch": 0.6906558328496808,
+      "grad_norm": 0.026488851755857468,
+      "learning_rate": 3.2762836185819075e-05,
+      "loss": 0.4479,
+      "step": 595
+    },
+    {
+      "epoch": 0.6964596633778294,
+      "grad_norm": 0.03377359360456467,
+      "learning_rate": 3.2151589242053795e-05,
+      "loss": 0.4628,
+      "step": 600
+    },
+    {
+      "epoch": 0.702263493905978,
+      "grad_norm": 0.03289955481886864,
+      "learning_rate": 3.154034229828851e-05,
+      "loss": 0.4332,
+      "step": 605
+    },
+    {
+      "epoch": 0.7080673244341266,
+      "grad_norm": 0.030403293669223785,
+      "learning_rate": 3.092909535452323e-05,
+      "loss": 0.4618,
+      "step": 610
+    },
+    {
+      "epoch": 0.7138711549622752,
+      "grad_norm": 0.03267165273427963,
+      "learning_rate": 3.031784841075795e-05,
+      "loss": 0.4536,
+      "step": 615
+    },
+    {
+      "epoch": 0.7196749854904236,
+      "grad_norm": 0.02854629047214985,
+      "learning_rate": 2.9706601466992666e-05,
+      "loss": 0.4405,
+      "step": 620
+    },
+    {
+      "epoch": 0.7254788160185722,
+      "grad_norm": 0.05602634325623512,
+      "learning_rate": 2.9095354523227386e-05,
+      "loss": 0.4272,
+      "step": 625
+    },
+    {
+      "epoch": 0.7312826465467208,
+      "grad_norm": 0.03450525179505348,
+      "learning_rate": 2.8484107579462103e-05,
+      "loss": 0.444,
+      "step": 630
+    },
+    {
+      "epoch": 0.7370864770748694,
+      "grad_norm": 0.029691684991121292,
+      "learning_rate": 2.7872860635696824e-05,
+      "loss": 0.4404,
+      "step": 635
+    },
+    {
+      "epoch": 0.742890307603018,
+      "grad_norm": 0.030143791809678078,
+      "learning_rate": 2.7261613691931544e-05,
+      "loss": 0.4586,
+      "step": 640
+    },
+    {
+      "epoch": 0.7486941381311666,
+      "grad_norm": 0.03883569687604904,
+      "learning_rate": 2.665036674816626e-05,
+      "loss": 0.4706,
+      "step": 645
+    },
+    {
+      "epoch": 0.7544979686593152,
+      "grad_norm": 0.03318801894783974,
+      "learning_rate": 2.603911980440098e-05,
+      "loss": 0.4526,
+      "step": 650
+    },
+    {
+      "epoch": 0.7603017991874638,
+      "grad_norm": 0.033630404621362686,
+      "learning_rate": 2.5427872860635698e-05,
+      "loss": 0.4289,
+      "step": 655
+    },
+    {
+      "epoch": 0.7661056297156124,
+      "grad_norm": 0.029941121116280556,
+      "learning_rate": 2.4816625916870418e-05,
+      "loss": 0.4409,
+      "step": 660
+    },
+    {
+      "epoch": 0.7719094602437608,
+      "grad_norm": 0.029758257791399956,
+      "learning_rate": 2.4205378973105135e-05,
+      "loss": 0.5286,
+      "step": 665
+    },
+    {
+      "epoch": 0.7777132907719094,
+      "grad_norm": 0.027401521801948547,
+      "learning_rate": 2.3594132029339855e-05,
+      "loss": 0.4292,
+      "step": 670
+    },
+    {
+      "epoch": 0.783517121300058,
+      "grad_norm": 0.03536379337310791,
+      "learning_rate": 2.2982885085574575e-05,
+      "loss": 0.421,
+      "step": 675
+    },
+    {
+      "epoch": 0.7893209518282066,
+      "grad_norm": 0.03377699479460716,
+      "learning_rate": 2.2371638141809292e-05,
+      "loss": 0.4425,
+      "step": 680
+    },
+    {
+      "epoch": 0.7951247823563552,
+      "grad_norm": 0.030305908992886543,
+      "learning_rate": 2.1760391198044012e-05,
+      "loss": 0.4618,
+      "step": 685
+    },
+    {
+      "epoch": 0.8009286128845038,
+      "grad_norm": 0.03241952136158943,
+      "learning_rate": 2.114914425427873e-05,
+      "loss": 0.4363,
+      "step": 690
+    },
+    {
+      "epoch": 0.8067324434126524,
+      "grad_norm": 0.02781447395682335,
+      "learning_rate": 2.053789731051345e-05,
+      "loss": 0.4608,
+      "step": 695
+    },
+    {
+      "epoch": 0.812536273940801,
+      "grad_norm": 0.03544509783387184,
+      "learning_rate": 1.9926650366748166e-05,
+      "loss": 0.4654,
+      "step": 700
+    },
+    {
+      "epoch": 0.8183401044689496,
+      "grad_norm": 0.038084760308265686,
+      "learning_rate": 1.9315403422982887e-05,
+      "loss": 0.4351,
+      "step": 705
+    },
+    {
+      "epoch": 0.824143934997098,
+      "grad_norm": 0.028816746547818184,
+      "learning_rate": 1.8704156479217603e-05,
+      "loss": 0.4395,
+      "step": 710
+    },
+    {
+      "epoch": 0.8299477655252466,
+      "grad_norm": 0.03324240446090698,
+      "learning_rate": 1.8092909535452324e-05,
+      "loss": 0.4553,
+      "step": 715
+    },
+    {
+      "epoch": 0.8357515960533952,
+      "grad_norm": 0.03481074050068855,
+      "learning_rate": 1.7481662591687044e-05,
+      "loss": 0.4651,
+      "step": 720
+    },
+    {
+      "epoch": 0.8415554265815438,
+      "grad_norm": 0.03580137714743614,
+      "learning_rate": 1.687041564792176e-05,
+      "loss": 0.4478,
+      "step": 725
+    },
+    {
+      "epoch": 0.8473592571096924,
+      "grad_norm": 0.03741241991519928,
+      "learning_rate": 1.625916870415648e-05,
+      "loss": 0.4384,
+      "step": 730
+    },
+    {
+      "epoch": 0.853163087637841,
+      "grad_norm": 0.04205469414591789,
+      "learning_rate": 1.5647921760391198e-05,
+      "loss": 0.4701,
+      "step": 735
+    },
+    {
+      "epoch": 0.8589669181659896,
+      "grad_norm": 0.027462556958198547,
+      "learning_rate": 1.5036674816625918e-05,
+      "loss": 0.4517,
+      "step": 740
+    },
+    {
+      "epoch": 0.8647707486941382,
+      "grad_norm": 0.023814866319298744,
+      "learning_rate": 1.4425427872860637e-05,
+      "loss": 0.4388,
+      "step": 745
+    },
+    {
+      "epoch": 0.8705745792222868,
+      "grad_norm": 0.033728573471307755,
+      "learning_rate": 1.3814180929095355e-05,
+      "loss": 0.4437,
+      "step": 750
+    },
+    {
+      "epoch": 0.8763784097504352,
+      "grad_norm": 0.03273741900920868,
+      "learning_rate": 1.3202933985330074e-05,
+      "loss": 0.4172,
+      "step": 755
+    },
+    {
+      "epoch": 0.8821822402785838,
+      "grad_norm": 0.03128568083047867,
+      "learning_rate": 1.2591687041564792e-05,
+      "loss": 0.4471,
+      "step": 760
+    },
+    {
+      "epoch": 0.8879860708067324,
+      "grad_norm": 0.03339388594031334,
+      "learning_rate": 1.198044009779951e-05,
+      "loss": 0.5063,
+      "step": 765
+    },
+    {
+      "epoch": 0.893789901334881,
+      "grad_norm": 0.036426227539777756,
+      "learning_rate": 1.136919315403423e-05,
+      "loss": 0.4416,
+      "step": 770
+    },
+    {
+      "epoch": 0.8995937318630296,
+      "grad_norm": 0.03400072082877159,
+      "learning_rate": 1.075794621026895e-05,
+      "loss": 0.4452,
+      "step": 775
+    },
+    {
+      "epoch": 0.9053975623911782,
+      "grad_norm": 0.04291456937789917,
+      "learning_rate": 1.0146699266503668e-05,
+      "loss": 0.429,
+      "step": 780
+    },
+    {
+      "epoch": 0.9112013929193268,
+      "grad_norm": 0.05733763426542282,
+      "learning_rate": 9.535452322738387e-06,
+      "loss": 0.4517,
+      "step": 785
+    },
+    {
+      "epoch": 0.9170052234474754,
+      "grad_norm": 0.027448438107967377,
+      "learning_rate": 8.924205378973105e-06,
+      "loss": 0.4384,
+      "step": 790
+    },
+    {
+      "epoch": 0.922809053975624,
+      "grad_norm": 0.03056074120104313,
+      "learning_rate": 8.312958435207824e-06,
+      "loss": 0.4504,
+      "step": 795
+    },
+    {
+      "epoch": 0.9286128845037724,
+      "grad_norm": 0.03468090295791626,
+      "learning_rate": 7.701711491442542e-06,
+      "loss": 0.4374,
+      "step": 800
+    },
+    {
+      "epoch": 0.934416715031921,
+      "grad_norm": 0.03324183076620102,
+      "learning_rate": 7.090464547677262e-06,
+      "loss": 0.4081,
+      "step": 805
+    },
+    {
+      "epoch": 0.9402205455600696,
+      "grad_norm": 0.028507592156529427,
+      "learning_rate": 6.47921760391198e-06,
+      "loss": 0.4296,
+      "step": 810
+    },
+    {
+      "epoch": 0.9460243760882182,
+      "grad_norm": 0.10197537392377853,
+      "learning_rate": 5.8679706601467e-06,
+      "loss": 0.4711,
+      "step": 815
+    },
+    {
+      "epoch": 0.9518282066163668,
+      "grad_norm": 0.02862210012972355,
+      "learning_rate": 5.256723716381418e-06,
+      "loss": 0.4413,
+      "step": 820
+    },
+    {
+      "epoch": 0.9576320371445154,
+      "grad_norm": 0.032685745507478714,
+      "learning_rate": 4.645476772616137e-06,
+      "loss": 0.4475,
+      "step": 825
+    },
+    {
+      "epoch": 0.963435867672664,
+      "grad_norm": 0.0329858735203743,
+      "learning_rate": 4.034229828850856e-06,
+      "loss": 0.42,
+      "step": 830
+    },
+    {
+      "epoch": 0.9692396982008126,
+      "grad_norm": 0.03359484672546387,
+      "learning_rate": 3.4229828850855746e-06,
+      "loss": 0.4499,
+      "step": 835
+    },
+    {
+      "epoch": 0.9750435287289612,
+      "grad_norm": 0.026256995275616646,
+      "learning_rate": 2.8117359413202936e-06,
+      "loss": 0.4475,
+      "step": 840
+    },
+    {
+      "epoch": 0.9808473592571096,
+      "grad_norm": 0.04073750227689743,
+      "learning_rate": 2.2004889975550126e-06,
+      "loss": 0.4509,
+      "step": 845
+    },
+    {
+      "epoch": 0.9866511897852582,
+      "grad_norm": 0.03341585770249367,
+      "learning_rate": 1.5892420537897311e-06,
+      "loss": 0.46,
+      "step": 850
+    },
+    {
+      "epoch": 0.9924550203134068,
+      "grad_norm": 0.0318162627518177,
+      "learning_rate": 9.779951100244499e-07,
+      "loss": 0.4118,
+      "step": 855
+    },
+    {
+      "epoch": 0.9982588508415554,
+      "grad_norm": 0.037213265895843506,
+      "learning_rate": 3.667481662591687e-07,
+      "loss": 0.4423,
+      "step": 860
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 862,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.192258616665068e+20,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Qwen3-4B-Instruct-2507-sft-fusang-swa=2k_sink=0/checkpoint-1302/README.md

@@ -0,0 +1,207 @@
+---
+base_model: /share/models/Qwen3-4B-Instruct-2507
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/share/models/Qwen3-4B-Instruct-2507
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1

Qwen3-4B-Instruct-2507-sft-fusang-swa=2k_sink=0/checkpoint-1302/adapter_config.json

@@ -0,0 +1,38 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/share/models/Qwen3-4B-Instruct-2507",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj",
+    "k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": true
+}

Qwen3-4B-Instruct-2507-sft-fusang-swa=2k_sink=0/checkpoint-1302/added_tokens.json

@@ -0,0 +1,28 @@
+{
+  "</think>": 151668,
+  "</tool_call>": 151658,
+  "</tool_response>": 151666,
+  "<think>": 151667,
+  "<tool_call>": 151657,
+  "<tool_response>": 151665,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

Qwen3-4B-Instruct-2507-sft-fusang-swa=2k_sink=0/checkpoint-1302/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Qwen3-4B-Instruct-2507-sft-fusang-swa=2k_sink=0/checkpoint-1302/trainer_state.json

@@ -0,0 +1,1854 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 6.0,
+  "eval_steps": 500,
+  "global_step": 1302,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.023108030040439053,
+      "grad_norm": 1.1153746843338013,
+      "learning_rate": 6.060606060606061e-06,
+      "loss": 0.3447,
+      "step": 5
+    },
+    {
+      "epoch": 0.046216060080878106,
+      "grad_norm": 1.1742913722991943,
+      "learning_rate": 1.3636363636363637e-05,
+      "loss": 0.3348,
+      "step": 10
+    },
+    {
+      "epoch": 0.06932409012131716,
+      "grad_norm": 0.979378879070282,
+      "learning_rate": 2.1212121212121215e-05,
+      "loss": 0.304,
+      "step": 15
+    },
+    {
+      "epoch": 0.09243212016175621,
+      "grad_norm": 0.8059154748916626,
+      "learning_rate": 2.878787878787879e-05,
+      "loss": 0.2841,
+      "step": 20
+    },
+    {
+      "epoch": 0.11554015020219527,
+      "grad_norm": 1.0525795221328735,
+      "learning_rate": 3.6363636363636364e-05,
+      "loss": 0.2677,
+      "step": 25
+    },
+    {
+      "epoch": 0.1386481802426343,
+      "grad_norm": 0.8975639939308167,
+      "learning_rate": 4.3939393939393944e-05,
+      "loss": 0.2502,
+      "step": 30
+    },
+    {
+      "epoch": 0.16175621028307338,
+      "grad_norm": 0.5892140865325928,
+      "learning_rate": 5.151515151515152e-05,
+      "loss": 0.2538,
+      "step": 35
+    },
+    {
+      "epoch": 0.18486424032351242,
+      "grad_norm": 0.67179936170578,
+      "learning_rate": 5.90909090909091e-05,
+      "loss": 0.2254,
+      "step": 40
+    },
+    {
+      "epoch": 0.20797227036395147,
+      "grad_norm": 0.6681568026542664,
+      "learning_rate": 6.666666666666667e-05,
+      "loss": 0.2328,
+      "step": 45
+    },
+    {
+      "epoch": 0.23108030040439054,
+      "grad_norm": 0.6512366533279419,
+      "learning_rate": 7.424242424242424e-05,
+      "loss": 0.2289,
+      "step": 50
+    },
+    {
+      "epoch": 0.2541883304448296,
+      "grad_norm": 0.8759204149246216,
+      "learning_rate": 8.181818181818183e-05,
+      "loss": 0.2147,
+      "step": 55
+    },
+    {
+      "epoch": 0.2772963604852686,
+      "grad_norm": 0.6086293458938599,
+      "learning_rate": 8.93939393939394e-05,
+      "loss": 0.219,
+      "step": 60
+    },
+    {
+      "epoch": 0.30040439052570767,
+      "grad_norm": 0.7560722827911377,
+      "learning_rate": 9.696969696969698e-05,
+      "loss": 0.1966,
+      "step": 65
+    },
+    {
+      "epoch": 0.32351242056614676,
+      "grad_norm": 0.5505369305610657,
+      "learning_rate": 9.975728155339807e-05,
+      "loss": 0.2055,
+      "step": 70
+    },
+    {
+      "epoch": 0.3466204506065858,
+      "grad_norm": 0.5759308934211731,
+      "learning_rate": 9.93527508090615e-05,
+      "loss": 0.2099,
+      "step": 75
+    },
+    {
+      "epoch": 0.36972848064702485,
+      "grad_norm": 0.5276843905448914,
+      "learning_rate": 9.894822006472492e-05,
+      "loss": 0.2051,
+      "step": 80
+    },
+    {
+      "epoch": 0.3928365106874639,
+      "grad_norm": 0.8748167157173157,
+      "learning_rate": 9.854368932038835e-05,
+      "loss": 0.2025,
+      "step": 85
+    },
+    {
+      "epoch": 0.41594454072790293,
+      "grad_norm": 0.5287477970123291,
+      "learning_rate": 9.813915857605178e-05,
+      "loss": 0.2005,
+      "step": 90
+    },
+    {
+      "epoch": 0.439052570768342,
+      "grad_norm": 0.5178080797195435,
+      "learning_rate": 9.773462783171522e-05,
+      "loss": 0.1933,
+      "step": 95
+    },
+    {
+      "epoch": 0.4621606008087811,
+      "grad_norm": 0.43820688128471375,
+      "learning_rate": 9.733009708737864e-05,
+      "loss": 0.1999,
+      "step": 100
+    },
+    {
+      "epoch": 0.4852686308492201,
+      "grad_norm": 0.7176827192306519,
+      "learning_rate": 9.692556634304207e-05,
+      "loss": 0.1943,
+      "step": 105
+    },
+    {
+      "epoch": 0.5083766608896592,
+      "grad_norm": 0.5665314197540283,
+      "learning_rate": 9.652103559870551e-05,
+      "loss": 0.2014,
+      "step": 110
+    },
+    {
+      "epoch": 0.5314846909300982,
+      "grad_norm": 0.5296651124954224,
+      "learning_rate": 9.611650485436893e-05,
+      "loss": 0.1824,
+      "step": 115
+    },
+    {
+      "epoch": 0.5545927209705372,
+      "grad_norm": 0.6124880313873291,
+      "learning_rate": 9.571197411003236e-05,
+      "loss": 0.1974,
+      "step": 120
+    },
+    {
+      "epoch": 0.5777007510109763,
+      "grad_norm": 0.4831707775592804,
+      "learning_rate": 9.530744336569579e-05,
+      "loss": 0.1946,
+      "step": 125
+    },
+    {
+      "epoch": 0.6008087810514153,
+      "grad_norm": 0.6636160016059875,
+      "learning_rate": 9.490291262135923e-05,
+      "loss": 0.1872,
+      "step": 130
+    },
+    {
+      "epoch": 0.6239168110918544,
+      "grad_norm": 0.5209525227546692,
+      "learning_rate": 9.449838187702265e-05,
+      "loss": 0.1904,
+      "step": 135
+    },
+    {
+      "epoch": 0.6470248411322935,
+      "grad_norm": 0.6263533234596252,
+      "learning_rate": 9.409385113268608e-05,
+      "loss": 0.1943,
+      "step": 140
+    },
+    {
+      "epoch": 0.6701328711727326,
+      "grad_norm": 0.8000551462173462,
+      "learning_rate": 9.368932038834952e-05,
+      "loss": 0.1921,
+      "step": 145
+    },
+    {
+      "epoch": 0.6932409012131716,
+      "grad_norm": 0.5430952310562134,
+      "learning_rate": 9.328478964401295e-05,
+      "loss": 0.1907,
+      "step": 150
+    },
+    {
+      "epoch": 0.7163489312536107,
+      "grad_norm": 0.4168851375579834,
+      "learning_rate": 9.288025889967637e-05,
+      "loss": 0.1825,
+      "step": 155
+    },
+    {
+      "epoch": 0.7394569612940497,
+      "grad_norm": 0.5036696791648865,
+      "learning_rate": 9.247572815533981e-05,
+      "loss": 0.1915,
+      "step": 160
+    },
+    {
+      "epoch": 0.7625649913344887,
+      "grad_norm": 0.5539006590843201,
+      "learning_rate": 9.207119741100324e-05,
+      "loss": 0.1852,
+      "step": 165
+    },
+    {
+      "epoch": 0.7856730213749278,
+      "grad_norm": 0.4744667708873749,
+      "learning_rate": 9.166666666666667e-05,
+      "loss": 0.1836,
+      "step": 170
+    },
+    {
+      "epoch": 0.8087810514153668,
+      "grad_norm": 0.4021014869213104,
+      "learning_rate": 9.126213592233011e-05,
+      "loss": 0.1866,
+      "step": 175
+    },
+    {
+      "epoch": 0.8318890814558059,
+      "grad_norm": 0.4380956292152405,
+      "learning_rate": 9.085760517799353e-05,
+      "loss": 0.1859,
+      "step": 180
+    },
+    {
+      "epoch": 0.8549971114962449,
+      "grad_norm": 0.41844603419303894,
+      "learning_rate": 9.045307443365696e-05,
+      "loss": 0.1922,
+      "step": 185
+    },
+    {
+      "epoch": 0.878105141536684,
+      "grad_norm": 0.40172576904296875,
+      "learning_rate": 9.00485436893204e-05,
+      "loss": 0.1845,
+      "step": 190
+    },
+    {
+      "epoch": 0.901213171577123,
+      "grad_norm": 0.4789203405380249,
+      "learning_rate": 8.964401294498383e-05,
+      "loss": 0.1845,
+      "step": 195
+    },
+    {
+      "epoch": 0.9243212016175621,
+      "grad_norm": 0.8636481761932373,
+      "learning_rate": 8.923948220064725e-05,
+      "loss": 0.191,
+      "step": 200
+    },
+    {
+      "epoch": 0.9474292316580012,
+      "grad_norm": 0.4059838354587555,
+      "learning_rate": 8.88349514563107e-05,
+      "loss": 0.1824,
+      "step": 205
+    },
+    {
+      "epoch": 0.9705372616984402,
+      "grad_norm": 0.4362904131412506,
+      "learning_rate": 8.843042071197412e-05,
+      "loss": 0.1864,
+      "step": 210
+    },
+    {
+      "epoch": 0.9936452917388793,
+      "grad_norm": 0.4498308598995209,
+      "learning_rate": 8.802588996763755e-05,
+      "loss": 0.1853,
+      "step": 215
+    },
+    {
+      "epoch": 1.0138648180242635,
+      "grad_norm": 0.4270644783973694,
+      "learning_rate": 8.762135922330097e-05,
+      "loss": 0.1822,
+      "step": 220
+    },
+    {
+      "epoch": 1.0369728480647025,
+      "grad_norm": 1.0029226541519165,
+      "learning_rate": 8.721682847896441e-05,
+      "loss": 0.1714,
+      "step": 225
+    },
+    {
+      "epoch": 1.0600808781051416,
+      "grad_norm": 0.39952850341796875,
+      "learning_rate": 8.681229773462784e-05,
+      "loss": 0.1743,
+      "step": 230
+    },
+    {
+      "epoch": 1.0831889081455806,
+      "grad_norm": 0.39805012941360474,
+      "learning_rate": 8.640776699029127e-05,
+      "loss": 0.1691,
+      "step": 235
+    },
+    {
+      "epoch": 1.1062969381860197,
+      "grad_norm": 0.47441813349723816,
+      "learning_rate": 8.60032362459547e-05,
+      "loss": 0.1816,
+      "step": 240
+    },
+    {
+      "epoch": 1.1294049682264586,
+      "grad_norm": 0.5233753323554993,
+      "learning_rate": 8.559870550161812e-05,
+      "loss": 0.1644,
+      "step": 245
+    },
+    {
+      "epoch": 1.1525129982668978,
+      "grad_norm": 0.6288277506828308,
+      "learning_rate": 8.519417475728155e-05,
+      "loss": 0.1752,
+      "step": 250
+    },
+    {
+      "epoch": 1.1756210283073367,
+      "grad_norm": 0.4849056899547577,
+      "learning_rate": 8.478964401294499e-05,
+      "loss": 0.1629,
+      "step": 255
+    },
+    {
+      "epoch": 1.1987290583477759,
+      "grad_norm": 0.3590959906578064,
+      "learning_rate": 8.438511326860841e-05,
+      "loss": 0.1735,
+      "step": 260
+    },
+    {
+      "epoch": 1.2218370883882148,
+      "grad_norm": 0.3890710473060608,
+      "learning_rate": 8.398058252427184e-05,
+      "loss": 0.1675,
+      "step": 265
+    },
+    {
+      "epoch": 1.244945118428654,
+      "grad_norm": 0.41331058740615845,
+      "learning_rate": 8.357605177993528e-05,
+      "loss": 0.1676,
+      "step": 270
+    },
+    {
+      "epoch": 1.2680531484690931,
+      "grad_norm": 0.39606937766075134,
+      "learning_rate": 8.31715210355987e-05,
+      "loss": 0.1704,
+      "step": 275
+    },
+    {
+      "epoch": 1.291161178509532,
+      "grad_norm": 0.4638404846191406,
+      "learning_rate": 8.276699029126213e-05,
+      "loss": 0.1695,
+      "step": 280
+    },
+    {
+      "epoch": 1.314269208549971,
+      "grad_norm": 0.38421422243118286,
+      "learning_rate": 8.236245954692557e-05,
+      "loss": 0.1679,
+      "step": 285
+    },
+    {
+      "epoch": 1.3373772385904101,
+      "grad_norm": 0.4009819030761719,
+      "learning_rate": 8.1957928802589e-05,
+      "loss": 0.1665,
+      "step": 290
+    },
+    {
+      "epoch": 1.3604852686308493,
+      "grad_norm": 0.45295777916908264,
+      "learning_rate": 8.155339805825243e-05,
+      "loss": 0.167,
+      "step": 295
+    },
+    {
+      "epoch": 1.3835932986712882,
+      "grad_norm": 0.4330746829509735,
+      "learning_rate": 8.114886731391587e-05,
+      "loss": 0.1688,
+      "step": 300
+    },
+    {
+      "epoch": 1.4067013287117274,
+      "grad_norm": 0.462908536195755,
+      "learning_rate": 8.074433656957929e-05,
+      "loss": 0.165,
+      "step": 305
+    },
+    {
+      "epoch": 1.4298093587521663,
+      "grad_norm": 0.38244912028312683,
+      "learning_rate": 8.033980582524272e-05,
+      "loss": 0.1726,
+      "step": 310
+    },
+    {
+      "epoch": 1.4529173887926055,
+      "grad_norm": 0.5104561448097229,
+      "learning_rate": 7.993527508090616e-05,
+      "loss": 0.1693,
+      "step": 315
+    },
+    {
+      "epoch": 1.4760254188330446,
+      "grad_norm": 0.4214445650577545,
+      "learning_rate": 7.953074433656959e-05,
+      "loss": 0.1627,
+      "step": 320
+    },
+    {
+      "epoch": 1.4991334488734835,
+      "grad_norm": 0.4412940442562103,
+      "learning_rate": 7.912621359223301e-05,
+      "loss": 0.1683,
+      "step": 325
+    },
+    {
+      "epoch": 1.5222414789139225,
+      "grad_norm": 0.40069451928138733,
+      "learning_rate": 7.872168284789644e-05,
+      "loss": 0.1708,
+      "step": 330
+    },
+    {
+      "epoch": 1.5453495089543616,
+      "grad_norm": 0.49756890535354614,
+      "learning_rate": 7.831715210355988e-05,
+      "loss": 0.1658,
+      "step": 335
+    },
+    {
+      "epoch": 1.5684575389948008,
+      "grad_norm": 0.4639304578304291,
+      "learning_rate": 7.79126213592233e-05,
+      "loss": 0.172,
+      "step": 340
+    },
+    {
+      "epoch": 1.5915655690352397,
+      "grad_norm": 0.4946388602256775,
+      "learning_rate": 7.750809061488673e-05,
+      "loss": 0.1705,
+      "step": 345
+    },
+    {
+      "epoch": 1.6146735990756786,
+      "grad_norm": 0.4386902451515198,
+      "learning_rate": 7.710355987055017e-05,
+      "loss": 0.1712,
+      "step": 350
+    },
+    {
+      "epoch": 1.6377816291161178,
+      "grad_norm": 0.41191956400871277,
+      "learning_rate": 7.66990291262136e-05,
+      "loss": 0.1666,
+      "step": 355
+    },
+    {
+      "epoch": 1.660889659156557,
+      "grad_norm": 0.47700071334838867,
+      "learning_rate": 7.629449838187703e-05,
+      "loss": 0.1672,
+      "step": 360
+    },
+    {
+      "epoch": 1.683997689196996,
+      "grad_norm": 0.48070764541625977,
+      "learning_rate": 7.588996763754047e-05,
+      "loss": 0.1732,
+      "step": 365
+    },
+    {
+      "epoch": 1.707105719237435,
+      "grad_norm": 0.3824273943901062,
+      "learning_rate": 7.548543689320389e-05,
+      "loss": 0.1607,
+      "step": 370
+    },
+    {
+      "epoch": 1.730213749277874,
+      "grad_norm": 0.4066704213619232,
+      "learning_rate": 7.508090614886732e-05,
+      "loss": 0.1668,
+      "step": 375
+    },
+    {
+      "epoch": 1.7533217793183131,
+      "grad_norm": 0.47296902537345886,
+      "learning_rate": 7.467637540453075e-05,
+      "loss": 0.1669,
+      "step": 380
+    },
+    {
+      "epoch": 1.7764298093587523,
+      "grad_norm": 0.4034911096096039,
+      "learning_rate": 7.427184466019417e-05,
+      "loss": 0.1729,
+      "step": 385
+    },
+    {
+      "epoch": 1.7995378393991912,
+      "grad_norm": 0.4491581618785858,
+      "learning_rate": 7.386731391585761e-05,
+      "loss": 0.1753,
+      "step": 390
+    },
+    {
+      "epoch": 1.8226458694396301,
+      "grad_norm": 0.42625272274017334,
+      "learning_rate": 7.346278317152104e-05,
+      "loss": 0.1724,
+      "step": 395
+    },
+    {
+      "epoch": 1.8457538994800693,
+      "grad_norm": 0.39192551374435425,
+      "learning_rate": 7.305825242718447e-05,
+      "loss": 0.1672,
+      "step": 400
+    },
+    {
+      "epoch": 1.8688619295205084,
+      "grad_norm": 0.3907965421676636,
+      "learning_rate": 7.265372168284789e-05,
+      "loss": 0.1667,
+      "step": 405
+    },
+    {
+      "epoch": 1.8919699595609474,
+      "grad_norm": 0.41404327750205994,
+      "learning_rate": 7.224919093851133e-05,
+      "loss": 0.1653,
+      "step": 410
+    },
+    {
+      "epoch": 1.9150779896013865,
+      "grad_norm": 0.3602406084537506,
+      "learning_rate": 7.184466019417476e-05,
+      "loss": 0.162,
+      "step": 415
+    },
+    {
+      "epoch": 1.9381860196418255,
+      "grad_norm": 0.4015500247478485,
+      "learning_rate": 7.144012944983819e-05,
+      "loss": 0.1669,
+      "step": 420
+    },
+    {
+      "epoch": 1.9612940496822646,
+      "grad_norm": 0.4846267104148865,
+      "learning_rate": 7.103559870550163e-05,
+      "loss": 0.168,
+      "step": 425
+    },
+    {
+      "epoch": 1.9844020797227038,
+      "grad_norm": 0.4054913818836212,
+      "learning_rate": 7.063106796116505e-05,
+      "loss": 0.1745,
+      "step": 430
+    },
+    {
+      "epoch": 2.004621606008088,
+      "grad_norm": 0.3823396861553192,
+      "learning_rate": 7.022653721682848e-05,
+      "loss": 0.1619,
+      "step": 435
+    },
+    {
+      "epoch": 2.027729636048527,
+      "grad_norm": 0.32919931411743164,
+      "learning_rate": 6.98220064724919e-05,
+      "loss": 0.1501,
+      "step": 440
+    },
+    {
+      "epoch": 2.050837666088966,
+      "grad_norm": 0.40527135133743286,
+      "learning_rate": 6.941747572815534e-05,
+      "loss": 0.1487,
+      "step": 445
+    },
+    {
+      "epoch": 2.073945696129405,
+      "grad_norm": 0.4119425117969513,
+      "learning_rate": 6.901294498381877e-05,
+      "loss": 0.1509,
+      "step": 450
+    },
+    {
+      "epoch": 2.097053726169844,
+      "grad_norm": 0.3594697117805481,
+      "learning_rate": 6.86084142394822e-05,
+      "loss": 0.1533,
+      "step": 455
+    },
+    {
+      "epoch": 2.1201617562102832,
+      "grad_norm": 0.48130837082862854,
+      "learning_rate": 6.820388349514564e-05,
+      "loss": 0.1568,
+      "step": 460
+    },
+    {
+      "epoch": 2.143269786250722,
+      "grad_norm": 0.3265611529350281,
+      "learning_rate": 6.779935275080906e-05,
+      "loss": 0.1519,
+      "step": 465
+    },
+    {
+      "epoch": 2.166377816291161,
+      "grad_norm": 0.35191479325294495,
+      "learning_rate": 6.739482200647249e-05,
+      "loss": 0.1548,
+      "step": 470
+    },
+    {
+      "epoch": 2.1894858463316003,
+      "grad_norm": 0.452744722366333,
+      "learning_rate": 6.699029126213593e-05,
+      "loss": 0.1536,
+      "step": 475
+    },
+    {
+      "epoch": 2.2125938763720394,
+      "grad_norm": 0.45772039890289307,
+      "learning_rate": 6.658576051779936e-05,
+      "loss": 0.149,
+      "step": 480
+    },
+    {
+      "epoch": 2.235701906412478,
+      "grad_norm": 0.40825536847114563,
+      "learning_rate": 6.618122977346278e-05,
+      "loss": 0.1537,
+      "step": 485
+    },
+    {
+      "epoch": 2.2588099364529173,
+      "grad_norm": 0.3820662796497345,
+      "learning_rate": 6.577669902912622e-05,
+      "loss": 0.1545,
+      "step": 490
+    },
+    {
+      "epoch": 2.2819179664933564,
+      "grad_norm": 0.4177970886230469,
+      "learning_rate": 6.537216828478965e-05,
+      "loss": 0.1534,
+      "step": 495
+    },
+    {
+      "epoch": 2.3050259965337956,
+      "grad_norm": 0.39039430022239685,
+      "learning_rate": 6.496763754045308e-05,
+      "loss": 0.1496,
+      "step": 500
+    },
+    {
+      "epoch": 2.3281340265742347,
+      "grad_norm": 0.4548358619213104,
+      "learning_rate": 6.456310679611652e-05,
+      "loss": 0.1492,
+      "step": 505
+    },
+    {
+      "epoch": 2.3512420566146734,
+      "grad_norm": 0.42207637429237366,
+      "learning_rate": 6.415857605177994e-05,
+      "loss": 0.1509,
+      "step": 510
+    },
+    {
+      "epoch": 2.3743500866551126,
+      "grad_norm": 0.40377140045166016,
+      "learning_rate": 6.375404530744337e-05,
+      "loss": 0.1599,
+      "step": 515
+    },
+    {
+      "epoch": 2.3974581166955518,
+      "grad_norm": 0.4512903094291687,
+      "learning_rate": 6.334951456310681e-05,
+      "loss": 0.1513,
+      "step": 520
+    },
+    {
+      "epoch": 2.420566146735991,
+      "grad_norm": 0.4324026107788086,
+      "learning_rate": 6.294498381877024e-05,
+      "loss": 0.1533,
+      "step": 525
+    },
+    {
+      "epoch": 2.4436741767764296,
+      "grad_norm": 0.41282832622528076,
+      "learning_rate": 6.254045307443366e-05,
+      "loss": 0.1569,
+      "step": 530
+    },
+    {
+      "epoch": 2.4667822068168688,
+      "grad_norm": 0.47760000824928284,
+      "learning_rate": 6.213592233009709e-05,
+      "loss": 0.1515,
+      "step": 535
+    },
+    {
+      "epoch": 2.489890236857308,
+      "grad_norm": 0.4068601131439209,
+      "learning_rate": 6.173139158576052e-05,
+      "loss": 0.1543,
+      "step": 540
+    },
+    {
+      "epoch": 2.512998266897747,
+      "grad_norm": 0.40145573019981384,
+      "learning_rate": 6.132686084142394e-05,
+      "loss": 0.157,
+      "step": 545
+    },
+    {
+      "epoch": 2.5361062969381862,
+      "grad_norm": 0.5375075936317444,
+      "learning_rate": 6.092233009708738e-05,
+      "loss": 0.1615,
+      "step": 550
+    },
+    {
+      "epoch": 2.559214326978625,
+      "grad_norm": 0.474707692861557,
+      "learning_rate": 6.051779935275082e-05,
+      "loss": 0.1518,
+      "step": 555
+    },
+    {
+      "epoch": 2.582322357019064,
+      "grad_norm": 0.44300198554992676,
+      "learning_rate": 6.0113268608414244e-05,
+      "loss": 0.158,
+      "step": 560
+    },
+    {
+      "epoch": 2.6054303870595032,
+      "grad_norm": 0.33379092812538147,
+      "learning_rate": 5.970873786407767e-05,
+      "loss": 0.1577,
+      "step": 565
+    },
+    {
+      "epoch": 2.628538417099942,
+      "grad_norm": 0.44380903244018555,
+      "learning_rate": 5.9304207119741104e-05,
+      "loss": 0.1456,
+      "step": 570
+    },
+    {
+      "epoch": 2.651646447140381,
+      "grad_norm": 0.38497334718704224,
+      "learning_rate": 5.889967637540453e-05,
+      "loss": 0.16,
+      "step": 575
+    },
+    {
+      "epoch": 2.6747544771808203,
+      "grad_norm": 0.3369431495666504,
+      "learning_rate": 5.8495145631067963e-05,
+      "loss": 0.1556,
+      "step": 580
+    },
+    {
+      "epoch": 2.6978625072212594,
+      "grad_norm": 0.37464407086372375,
+      "learning_rate": 5.80906148867314e-05,
+      "loss": 0.1544,
+      "step": 585
+    },
+    {
+      "epoch": 2.7209705372616986,
+      "grad_norm": 0.4785277545452118,
+      "learning_rate": 5.768608414239482e-05,
+      "loss": 0.1528,
+      "step": 590
+    },
+    {
+      "epoch": 2.7440785673021377,
+      "grad_norm": 0.3807770609855652,
+      "learning_rate": 5.728155339805825e-05,
+      "loss": 0.1535,
+      "step": 595
+    },
+    {
+      "epoch": 2.7671865973425764,
+      "grad_norm": 0.41145429015159607,
+      "learning_rate": 5.687702265372169e-05,
+      "loss": 0.1564,
+      "step": 600
+    },
+    {
+      "epoch": 2.7902946273830156,
+      "grad_norm": 0.4205482602119446,
+      "learning_rate": 5.6472491909385117e-05,
+      "loss": 0.1534,
+      "step": 605
+    },
+    {
+      "epoch": 2.8134026574234547,
+      "grad_norm": 0.4658476710319519,
+      "learning_rate": 5.606796116504854e-05,
+      "loss": 0.1503,
+      "step": 610
+    },
+    {
+      "epoch": 2.8365106874638935,
+      "grad_norm": 0.33467110991477966,
+      "learning_rate": 5.566343042071198e-05,
+      "loss": 0.1595,
+      "step": 615
+    },
+    {
+      "epoch": 2.8596187175043326,
+      "grad_norm": 0.39783260226249695,
+      "learning_rate": 5.525889967637541e-05,
+      "loss": 0.1508,
+      "step": 620
+    },
+    {
+      "epoch": 2.8827267475447718,
+      "grad_norm": 0.41435226798057556,
+      "learning_rate": 5.4854368932038836e-05,
+      "loss": 0.1553,
+      "step": 625
+    },
+    {
+      "epoch": 2.905834777585211,
+      "grad_norm": 0.35281336307525635,
+      "learning_rate": 5.4449838187702276e-05,
+      "loss": 0.1489,
+      "step": 630
+    },
+    {
+      "epoch": 2.92894280762565,
+      "grad_norm": 0.5651142001152039,
+      "learning_rate": 5.40453074433657e-05,
+      "loss": 0.1528,
+      "step": 635
+    },
+    {
+      "epoch": 2.9520508376660892,
+      "grad_norm": 0.31643569469451904,
+      "learning_rate": 5.364077669902913e-05,
+      "loss": 0.153,
+      "step": 640
+    },
+    {
+      "epoch": 2.975158867706528,
+      "grad_norm": 0.4358139634132385,
+      "learning_rate": 5.3236245954692556e-05,
+      "loss": 0.1503,
+      "step": 645
+    },
+    {
+      "epoch": 2.998266897746967,
+      "grad_norm": 0.4005185067653656,
+      "learning_rate": 5.283171521035599e-05,
+      "loss": 0.1567,
+      "step": 650
+    },
+    {
+      "epoch": 3.0184864240323512,
+      "grad_norm": 0.39221566915512085,
+      "learning_rate": 5.2427184466019416e-05,
+      "loss": 0.1511,
+      "step": 655
+    },
+    {
+      "epoch": 3.0415944540727904,
+      "grad_norm": 0.37966302037239075,
+      "learning_rate": 5.202265372168284e-05,
+      "loss": 0.1466,
+      "step": 660
+    },
+    {
+      "epoch": 3.0647024841132295,
+      "grad_norm": 0.4152965247631073,
+      "learning_rate": 5.161812297734628e-05,
+      "loss": 0.1437,
+      "step": 665
+    },
+    {
+      "epoch": 3.0878105141536683,
+      "grad_norm": 0.3665558993816376,
+      "learning_rate": 5.121359223300971e-05,
+      "loss": 0.1421,
+      "step": 670
+    },
+    {
+      "epoch": 3.1109185441941074,
+      "grad_norm": 0.3718888759613037,
+      "learning_rate": 5.0809061488673136e-05,
+      "loss": 0.1463,
+      "step": 675
+    },
+    {
+      "epoch": 3.1340265742345466,
+      "grad_norm": 0.3988041877746582,
+      "learning_rate": 5.0404530744336576e-05,
+      "loss": 0.1389,
+      "step": 680
+    },
+    {
+      "epoch": 3.1571346042749857,
+      "grad_norm": 0.43306609988212585,
+      "learning_rate": 5e-05,
+      "loss": 0.1401,
+      "step": 685
+    },
+    {
+      "epoch": 3.1802426343154244,
+      "grad_norm": 0.43652892112731934,
+      "learning_rate": 4.9595469255663436e-05,
+      "loss": 0.1368,
+      "step": 690
+    },
+    {
+      "epoch": 3.2033506643558636,
+      "grad_norm": 0.33591899275779724,
+      "learning_rate": 4.919093851132686e-05,
+      "loss": 0.1401,
+      "step": 695
+    },
+    {
+      "epoch": 3.2264586943963027,
+      "grad_norm": 0.36130473017692566,
+      "learning_rate": 4.8786407766990296e-05,
+      "loss": 0.1437,
+      "step": 700
+    },
+    {
+      "epoch": 3.249566724436742,
+      "grad_norm": 0.3655638098716736,
+      "learning_rate": 4.838187702265373e-05,
+      "loss": 0.1338,
+      "step": 705
+    },
+    {
+      "epoch": 3.2726747544771806,
+      "grad_norm": 0.38180845975875854,
+      "learning_rate": 4.7977346278317155e-05,
+      "loss": 0.1414,
+      "step": 710
+    },
+    {
+      "epoch": 3.2957827845176197,
+      "grad_norm": 0.4078177511692047,
+      "learning_rate": 4.757281553398059e-05,
+      "loss": 0.1441,
+      "step": 715
+    },
+    {
+      "epoch": 3.318890814558059,
+      "grad_norm": 0.3439587950706482,
+      "learning_rate": 4.7168284789644015e-05,
+      "loss": 0.1447,
+      "step": 720
+    },
+    {
+      "epoch": 3.341998844598498,
+      "grad_norm": 0.3950468897819519,
+      "learning_rate": 4.676375404530744e-05,
+      "loss": 0.1396,
+      "step": 725
+    },
+    {
+      "epoch": 3.365106874638937,
+      "grad_norm": 0.3649514615535736,
+      "learning_rate": 4.6359223300970875e-05,
+      "loss": 0.1429,
+      "step": 730
+    },
+    {
+      "epoch": 3.388214904679376,
+      "grad_norm": 0.37156760692596436,
+      "learning_rate": 4.59546925566343e-05,
+      "loss": 0.143,
+      "step": 735
+    },
+    {
+      "epoch": 3.411322934719815,
+      "grad_norm": 0.49312466382980347,
+      "learning_rate": 4.5550161812297735e-05,
+      "loss": 0.1375,
+      "step": 740
+    },
+    {
+      "epoch": 3.4344309647602542,
+      "grad_norm": 0.3575374186038971,
+      "learning_rate": 4.514563106796117e-05,
+      "loss": 0.141,
+      "step": 745
+    },
+    {
+      "epoch": 3.4575389948006934,
+      "grad_norm": 0.4175109267234802,
+      "learning_rate": 4.4741100323624595e-05,
+      "loss": 0.1391,
+      "step": 750
+    },
+    {
+      "epoch": 3.480647024841132,
+      "grad_norm": 0.38424453139305115,
+      "learning_rate": 4.433656957928803e-05,
+      "loss": 0.1396,
+      "step": 755
+    },
+    {
+      "epoch": 3.5037550548815712,
+      "grad_norm": 0.42046743631362915,
+      "learning_rate": 4.393203883495146e-05,
+      "loss": 0.1506,
+      "step": 760
+    },
+    {
+      "epoch": 3.5268630849220104,
+      "grad_norm": 0.3649904131889343,
+      "learning_rate": 4.352750809061489e-05,
+      "loss": 0.1405,
+      "step": 765
+    },
+    {
+      "epoch": 3.5499711149624495,
+      "grad_norm": 0.35645848512649536,
+      "learning_rate": 4.312297734627832e-05,
+      "loss": 0.141,
+      "step": 770
+    },
+    {
+      "epoch": 3.5730791450028887,
+      "grad_norm": 0.3775937259197235,
+      "learning_rate": 4.271844660194175e-05,
+      "loss": 0.1443,
+      "step": 775
+    },
+    {
+      "epoch": 3.5961871750433274,
+      "grad_norm": 0.4760645925998688,
+      "learning_rate": 4.231391585760518e-05,
+      "loss": 0.1353,
+      "step": 780
+    },
+    {
+      "epoch": 3.6192952050837666,
+      "grad_norm": 0.37175118923187256,
+      "learning_rate": 4.1909385113268615e-05,
+      "loss": 0.146,
+      "step": 785
+    },
+    {
+      "epoch": 3.6424032351242057,
+      "grad_norm": 0.403616338968277,
+      "learning_rate": 4.150485436893204e-05,
+      "loss": 0.141,
+      "step": 790
+    },
+    {
+      "epoch": 3.665511265164645,
+      "grad_norm": 0.3865659534931183,
+      "learning_rate": 4.1100323624595475e-05,
+      "loss": 0.1442,
+      "step": 795
+    },
+    {
+      "epoch": 3.6886192952050836,
+      "grad_norm": 0.3601677417755127,
+      "learning_rate": 4.06957928802589e-05,
+      "loss": 0.1358,
+      "step": 800
+    },
+    {
+      "epoch": 3.7117273252455227,
+      "grad_norm": 0.441893070936203,
+      "learning_rate": 4.029126213592233e-05,
+      "loss": 0.1336,
+      "step": 805
+    },
+    {
+      "epoch": 3.734835355285962,
+      "grad_norm": 0.40766894817352295,
+      "learning_rate": 3.988673139158576e-05,
+      "loss": 0.1432,
+      "step": 810
+    },
+    {
+      "epoch": 3.757943385326401,
+      "grad_norm": 0.3878239393234253,
+      "learning_rate": 3.948220064724919e-05,
+      "loss": 0.1443,
+      "step": 815
+    },
+    {
+      "epoch": 3.78105141536684,
+      "grad_norm": 0.35221049189567566,
+      "learning_rate": 3.907766990291262e-05,
+      "loss": 0.1416,
+      "step": 820
+    },
+    {
+      "epoch": 3.804159445407279,
+      "grad_norm": 0.3420145511627197,
+      "learning_rate": 3.8673139158576054e-05,
+      "loss": 0.1368,
+      "step": 825
+    },
+    {
+      "epoch": 3.827267475447718,
+      "grad_norm": 0.4021627604961395,
+      "learning_rate": 3.826860841423948e-05,
+      "loss": 0.1462,
+      "step": 830
+    },
+    {
+      "epoch": 3.850375505488157,
+      "grad_norm": 0.6137374043464661,
+      "learning_rate": 3.7864077669902914e-05,
+      "loss": 0.1479,
+      "step": 835
+    },
+    {
+      "epoch": 3.873483535528596,
+      "grad_norm": 0.4098761975765228,
+      "learning_rate": 3.745954692556635e-05,
+      "loss": 0.1429,
+      "step": 840
+    },
+    {
+      "epoch": 3.896591565569035,
+      "grad_norm": 0.3785853981971741,
+      "learning_rate": 3.7055016181229774e-05,
+      "loss": 0.1445,
+      "step": 845
+    },
+    {
+      "epoch": 3.9196995956094742,
+      "grad_norm": 0.36502259969711304,
+      "learning_rate": 3.665048543689321e-05,
+      "loss": 0.1419,
+      "step": 850
+    },
+    {
+      "epoch": 3.9428076256499134,
+      "grad_norm": 0.43983909487724304,
+      "learning_rate": 3.624595469255664e-05,
+      "loss": 0.1464,
+      "step": 855
+    },
+    {
+      "epoch": 3.9659156556903525,
+      "grad_norm": 0.4203372299671173,
+      "learning_rate": 3.584142394822007e-05,
+      "loss": 0.1405,
+      "step": 860
+    },
+    {
+      "epoch": 3.9890236857307917,
+      "grad_norm": 0.39132270216941833,
+      "learning_rate": 3.54368932038835e-05,
+      "loss": 0.1376,
+      "step": 865
+    },
+    {
+      "epoch": 4.009243212016176,
+      "grad_norm": 0.3865406811237335,
+      "learning_rate": 3.503236245954693e-05,
+      "loss": 0.1406,
+      "step": 870
+    },
+    {
+      "epoch": 4.032351242056615,
+      "grad_norm": 0.40331369638442993,
+      "learning_rate": 3.4627831715210354e-05,
+      "loss": 0.1338,
+      "step": 875
+    },
+    {
+      "epoch": 4.055459272097054,
+      "grad_norm": 0.3296009302139282,
+      "learning_rate": 3.422330097087379e-05,
+      "loss": 0.1371,
+      "step": 880
+    },
+    {
+      "epoch": 4.078567302137492,
+      "grad_norm": 0.38344064354896545,
+      "learning_rate": 3.3818770226537214e-05,
+      "loss": 0.139,
+      "step": 885
+    },
+    {
+      "epoch": 4.101675332177932,
+      "grad_norm": 0.35297831892967224,
+      "learning_rate": 3.341423948220065e-05,
+      "loss": 0.137,
+      "step": 890
+    },
+    {
+      "epoch": 4.124783362218371,
+      "grad_norm": 0.4332777261734009,
+      "learning_rate": 3.300970873786408e-05,
+      "loss": 0.1361,
+      "step": 895
+    },
+    {
+      "epoch": 4.14789139225881,
+      "grad_norm": 0.4004818797111511,
+      "learning_rate": 3.260517799352751e-05,
+      "loss": 0.1358,
+      "step": 900
+    },
+    {
+      "epoch": 4.170999422299249,
+      "grad_norm": 0.35670897364616394,
+      "learning_rate": 3.220064724919094e-05,
+      "loss": 0.1237,
+      "step": 905
+    },
+    {
+      "epoch": 4.194107452339688,
+      "grad_norm": 0.3451295495033264,
+      "learning_rate": 3.1796116504854373e-05,
+      "loss": 0.1331,
+      "step": 910
+    },
+    {
+      "epoch": 4.217215482380127,
+      "grad_norm": 0.399081826210022,
+      "learning_rate": 3.13915857605178e-05,
+      "loss": 0.138,
+      "step": 915
+    },
+    {
+      "epoch": 4.2403235124205665,
+      "grad_norm": 0.4160274863243103,
+      "learning_rate": 3.098705501618123e-05,
+      "loss": 0.1365,
+      "step": 920
+    },
+    {
+      "epoch": 4.263431542461005,
+      "grad_norm": 0.5327185392379761,
+      "learning_rate": 3.058252427184466e-05,
+      "loss": 0.1298,
+      "step": 925
+    },
+    {
+      "epoch": 4.286539572501444,
+      "grad_norm": 0.33715537190437317,
+      "learning_rate": 3.0177993527508093e-05,
+      "loss": 0.1346,
+      "step": 930
+    },
+    {
+      "epoch": 4.309647602541883,
+      "grad_norm": 0.3837791085243225,
+      "learning_rate": 2.9773462783171523e-05,
+      "loss": 0.1231,
+      "step": 935
+    },
+    {
+      "epoch": 4.332755632582322,
+      "grad_norm": 0.435489684343338,
+      "learning_rate": 2.936893203883495e-05,
+      "loss": 0.1344,
+      "step": 940
+    },
+    {
+      "epoch": 4.355863662622761,
+      "grad_norm": 0.4198773503303528,
+      "learning_rate": 2.8964401294498383e-05,
+      "loss": 0.1376,
+      "step": 945
+    },
+    {
+      "epoch": 4.3789716926632005,
+      "grad_norm": 1.8017693758010864,
+      "learning_rate": 2.8559870550161816e-05,
+      "loss": 0.1389,
+      "step": 950
+    },
+    {
+      "epoch": 4.40207972270364,
+      "grad_norm": 0.336780309677124,
+      "learning_rate": 2.8155339805825243e-05,
+      "loss": 0.1315,
+      "step": 955
+    },
+    {
+      "epoch": 4.425187752744079,
+      "grad_norm": 0.4759547710418701,
+      "learning_rate": 2.7750809061488676e-05,
+      "loss": 0.1288,
+      "step": 960
+    },
+    {
+      "epoch": 4.448295782784518,
+      "grad_norm": 0.3436700105667114,
+      "learning_rate": 2.7346278317152106e-05,
+      "loss": 0.1411,
+      "step": 965
+    },
+    {
+      "epoch": 4.471403812824956,
+      "grad_norm": 0.35549604892730713,
+      "learning_rate": 2.6941747572815533e-05,
+      "loss": 0.1289,
+      "step": 970
+    },
+    {
+      "epoch": 4.494511842865395,
+      "grad_norm": 0.3846369683742523,
+      "learning_rate": 2.6537216828478966e-05,
+      "loss": 0.1345,
+      "step": 975
+    },
+    {
+      "epoch": 4.5176198729058346,
+      "grad_norm": 0.34032395482063293,
+      "learning_rate": 2.6132686084142393e-05,
+      "loss": 0.1335,
+      "step": 980
+    },
+    {
+      "epoch": 4.540727902946274,
+      "grad_norm": 0.37307125329971313,
+      "learning_rate": 2.5728155339805826e-05,
+      "loss": 0.1349,
+      "step": 985
+    },
+    {
+      "epoch": 4.563835932986713,
+      "grad_norm": 0.4150727689266205,
+      "learning_rate": 2.532362459546926e-05,
+      "loss": 0.1276,
+      "step": 990
+    },
+    {
+      "epoch": 4.586943963027152,
+      "grad_norm": 0.3694347143173218,
+      "learning_rate": 2.491909385113269e-05,
+      "loss": 0.1365,
+      "step": 995
+    },
+    {
+      "epoch": 4.610051993067591,
+      "grad_norm": 0.3570181727409363,
+      "learning_rate": 2.451456310679612e-05,
+      "loss": 0.1342,
+      "step": 1000
+    },
+    {
+      "epoch": 4.63316002310803,
+      "grad_norm": 0.3931976556777954,
+      "learning_rate": 2.411003236245955e-05,
+      "loss": 0.1357,
+      "step": 1005
+    },
+    {
+      "epoch": 4.6562680531484695,
+      "grad_norm": 0.3693641126155853,
+      "learning_rate": 2.3705501618122976e-05,
+      "loss": 0.1292,
+      "step": 1010
+    },
+    {
+      "epoch": 4.679376083188908,
+      "grad_norm": 0.3989200294017792,
+      "learning_rate": 2.330097087378641e-05,
+      "loss": 0.1403,
+      "step": 1015
+    },
+    {
+      "epoch": 4.702484113229347,
+      "grad_norm": 0.3670448362827301,
+      "learning_rate": 2.289644012944984e-05,
+      "loss": 0.1324,
+      "step": 1020
+    },
+    {
+      "epoch": 4.725592143269786,
+      "grad_norm": 0.4417153298854828,
+      "learning_rate": 2.249190938511327e-05,
+      "loss": 0.1311,
+      "step": 1025
+    },
+    {
+      "epoch": 4.748700173310225,
+      "grad_norm": 0.3508768081665039,
+      "learning_rate": 2.2087378640776702e-05,
+      "loss": 0.1307,
+      "step": 1030
+    },
+    {
+      "epoch": 4.771808203350664,
+      "grad_norm": 0.4192269742488861,
+      "learning_rate": 2.1682847896440132e-05,
+      "loss": 0.1316,
+      "step": 1035
+    },
+    {
+      "epoch": 4.7949162333911035,
+      "grad_norm": 0.4704461395740509,
+      "learning_rate": 2.1278317152103562e-05,
+      "loss": 0.1394,
+      "step": 1040
+    },
+    {
+      "epoch": 4.818024263431543,
+      "grad_norm": 0.4367867708206177,
+      "learning_rate": 2.0873786407766992e-05,
+      "loss": 0.132,
+      "step": 1045
+    },
+    {
+      "epoch": 4.841132293471982,
+      "grad_norm": 0.45858272910118103,
+      "learning_rate": 2.0469255663430422e-05,
+      "loss": 0.1344,
+      "step": 1050
+    },
+    {
+      "epoch": 4.864240323512421,
+      "grad_norm": 0.3659626543521881,
+      "learning_rate": 2.0064724919093852e-05,
+      "loss": 0.1301,
+      "step": 1055
+    },
+    {
+      "epoch": 4.887348353552859,
+      "grad_norm": 0.398194819688797,
+      "learning_rate": 1.9660194174757282e-05,
+      "loss": 0.1281,
+      "step": 1060
+    },
+    {
+      "epoch": 4.910456383593298,
+      "grad_norm": 0.4072405695915222,
+      "learning_rate": 1.9255663430420712e-05,
+      "loss": 0.1355,
+      "step": 1065
+    },
+    {
+      "epoch": 4.9335644136337375,
+      "grad_norm": 0.36236122250556946,
+      "learning_rate": 1.8851132686084145e-05,
+      "loss": 0.1307,
+      "step": 1070
+    },
+    {
+      "epoch": 4.956672443674177,
+      "grad_norm": 0.40087103843688965,
+      "learning_rate": 1.8446601941747575e-05,
+      "loss": 0.1348,
+      "step": 1075
+    },
+    {
+      "epoch": 4.979780473714616,
+      "grad_norm": 0.3166773319244385,
+      "learning_rate": 1.8042071197411005e-05,
+      "loss": 0.1372,
+      "step": 1080
+    },
+    {
+      "epoch": 5.0,
+      "grad_norm": 0.4858568012714386,
+      "learning_rate": 1.763754045307443e-05,
+      "loss": 0.1295,
+      "step": 1085
+    },
+    {
+      "epoch": 5.023108030040439,
+      "grad_norm": 0.34639063477516174,
+      "learning_rate": 1.7233009708737865e-05,
+      "loss": 0.129,
+      "step": 1090
+    },
+    {
+      "epoch": 5.046216060080878,
+      "grad_norm": 0.34696677327156067,
+      "learning_rate": 1.6828478964401295e-05,
+      "loss": 0.1303,
+      "step": 1095
+    },
+    {
+      "epoch": 5.0693240901213175,
+      "grad_norm": 0.36168208718299866,
+      "learning_rate": 1.6423948220064725e-05,
+      "loss": 0.1305,
+      "step": 1100
+    },
+    {
+      "epoch": 5.092432120161757,
+      "grad_norm": 0.40556296706199646,
+      "learning_rate": 1.6019417475728158e-05,
+      "loss": 0.1304,
+      "step": 1105
+    },
+    {
+      "epoch": 5.115540150202195,
+      "grad_norm": 0.3203165829181671,
+      "learning_rate": 1.5614886731391588e-05,
+      "loss": 0.1244,
+      "step": 1110
+    },
+    {
+      "epoch": 5.138648180242634,
+      "grad_norm": 0.3708655536174774,
+      "learning_rate": 1.5210355987055016e-05,
+      "loss": 0.1243,
+      "step": 1115
+    },
+    {
+      "epoch": 5.161756210283073,
+      "grad_norm": 0.44016334414482117,
+      "learning_rate": 1.4805825242718446e-05,
+      "loss": 0.1253,
+      "step": 1120
+    },
+    {
+      "epoch": 5.184864240323512,
+      "grad_norm": 0.3627468943595886,
+      "learning_rate": 1.440129449838188e-05,
+      "loss": 0.1243,
+      "step": 1125
+    },
+    {
+      "epoch": 5.2079722703639515,
+      "grad_norm": 0.42211201786994934,
+      "learning_rate": 1.3996763754045308e-05,
+      "loss": 0.1264,
+      "step": 1130
+    },
+    {
+      "epoch": 5.231080300404391,
+      "grad_norm": 0.37269696593284607,
+      "learning_rate": 1.3592233009708738e-05,
+      "loss": 0.1334,
+      "step": 1135
+    },
+    {
+      "epoch": 5.25418833044483,
+      "grad_norm": 0.35278457403182983,
+      "learning_rate": 1.3187702265372168e-05,
+      "loss": 0.1151,
+      "step": 1140
+    },
+    {
+      "epoch": 5.277296360485269,
+      "grad_norm": 0.3814377188682556,
+      "learning_rate": 1.27831715210356e-05,
+      "loss": 0.1208,
+      "step": 1145
+    },
+    {
+      "epoch": 5.300404390525708,
+      "grad_norm": 0.3895890712738037,
+      "learning_rate": 1.237864077669903e-05,
+      "loss": 0.1298,
+      "step": 1150
+    },
+    {
+      "epoch": 5.323512420566146,
+      "grad_norm": 0.3315405547618866,
+      "learning_rate": 1.197411003236246e-05,
+      "loss": 0.1234,
+      "step": 1155
+    },
+    {
+      "epoch": 5.3466204506065855,
+      "grad_norm": 0.2715717554092407,
+      "learning_rate": 1.156957928802589e-05,
+      "loss": 0.1357,
+      "step": 1160
+    },
+    {
+      "epoch": 5.369728480647025,
+      "grad_norm": 0.38255131244659424,
+      "learning_rate": 1.116504854368932e-05,
+      "loss": 0.1231,
+      "step": 1165
+    },
+    {
+      "epoch": 5.392836510687464,
+      "grad_norm": 0.7304874062538147,
+      "learning_rate": 1.076051779935275e-05,
+      "loss": 0.1278,
+      "step": 1170
+    },
+    {
+      "epoch": 5.415944540727903,
+      "grad_norm": 0.44462937116622925,
+      "learning_rate": 1.0355987055016182e-05,
+      "loss": 0.1355,
+      "step": 1175
+    },
+    {
+      "epoch": 5.439052570768342,
+      "grad_norm": 0.37233683466911316,
+      "learning_rate": 9.951456310679612e-06,
+      "loss": 0.1256,
+      "step": 1180
+    },
+    {
+      "epoch": 5.462160600808781,
+      "grad_norm": 0.3165900409221649,
+      "learning_rate": 9.546925566343042e-06,
+      "loss": 0.1297,
+      "step": 1185
+    },
+    {
+      "epoch": 5.4852686308492205,
+      "grad_norm": 0.3394305109977722,
+      "learning_rate": 9.142394822006472e-06,
+      "loss": 0.1259,
+      "step": 1190
+    },
+    {
+      "epoch": 5.50837666088966,
+      "grad_norm": 0.35609307885169983,
+      "learning_rate": 8.737864077669904e-06,
+      "loss": 0.1328,
+      "step": 1195
+    },
+    {
+      "epoch": 5.531484690930098,
+      "grad_norm": 0.42052727937698364,
+      "learning_rate": 8.333333333333334e-06,
+      "loss": 0.1307,
+      "step": 1200
+    },
+    {
+      "epoch": 5.554592720970537,
+      "grad_norm": 0.4544830322265625,
+      "learning_rate": 7.928802588996764e-06,
+      "loss": 0.1285,
+      "step": 1205
+    },
+    {
+      "epoch": 5.577700751010976,
+      "grad_norm": 0.3438180387020111,
+      "learning_rate": 7.524271844660194e-06,
+      "loss": 0.129,
+      "step": 1210
+    },
+    {
+      "epoch": 5.600808781051415,
+      "grad_norm": 0.33500605821609497,
+      "learning_rate": 7.119741100323625e-06,
+      "loss": 0.1225,
+      "step": 1215
+    },
+    {
+      "epoch": 5.6239168110918545,
+      "grad_norm": 0.36368703842163086,
+      "learning_rate": 6.715210355987054e-06,
+      "loss": 0.1309,
+      "step": 1220
+    },
+    {
+      "epoch": 5.647024841132294,
+      "grad_norm": 0.3638695180416107,
+      "learning_rate": 6.310679611650486e-06,
+      "loss": 0.1222,
+      "step": 1225
+    },
+    {
+      "epoch": 5.670132871172733,
+      "grad_norm": 0.4183124899864197,
+      "learning_rate": 5.906148867313916e-06,
+      "loss": 0.1259,
+      "step": 1230
+    },
+    {
+      "epoch": 5.693240901213172,
+      "grad_norm": 0.46720147132873535,
+      "learning_rate": 5.501618122977347e-06,
+      "loss": 0.1281,
+      "step": 1235
+    },
+    {
+      "epoch": 5.716348931253611,
+      "grad_norm": 0.32759833335876465,
+      "learning_rate": 5.097087378640777e-06,
+      "loss": 0.1225,
+      "step": 1240
+    },
+    {
+      "epoch": 5.739456961294049,
+      "grad_norm": 0.37824955582618713,
+      "learning_rate": 4.6925566343042074e-06,
+      "loss": 0.1223,
+      "step": 1245
+    },
+    {
+      "epoch": 5.7625649913344885,
+      "grad_norm": 0.33378538489341736,
+      "learning_rate": 4.288025889967637e-06,
+      "loss": 0.1323,
+      "step": 1250
+    },
+    {
+      "epoch": 5.785673021374928,
+      "grad_norm": 0.3983270525932312,
+      "learning_rate": 3.883495145631068e-06,
+      "loss": 0.1261,
+      "step": 1255
+    },
+    {
+      "epoch": 5.808781051415367,
+      "grad_norm": 0.324893981218338,
+      "learning_rate": 3.478964401294499e-06,
+      "loss": 0.1301,
+      "step": 1260
+    },
+    {
+      "epoch": 5.831889081455806,
+      "grad_norm": 0.33863502740859985,
+      "learning_rate": 3.074433656957929e-06,
+      "loss": 0.1268,
+      "step": 1265
+    },
+    {
+      "epoch": 5.854997111496245,
+      "grad_norm": 0.3771996796131134,
+      "learning_rate": 2.6699029126213593e-06,
+      "loss": 0.1334,
+      "step": 1270
+    },
+    {
+      "epoch": 5.878105141536684,
+      "grad_norm": 0.406698077917099,
+      "learning_rate": 2.26537216828479e-06,
+      "loss": 0.1322,
+      "step": 1275
+    },
+    {
+      "epoch": 5.9012131715771226,
+      "grad_norm": 0.4297156035900116,
+      "learning_rate": 1.8608414239482202e-06,
+      "loss": 0.1312,
+      "step": 1280
+    },
+    {
+      "epoch": 5.924321201617563,
+      "grad_norm": 0.3260704278945923,
+      "learning_rate": 1.4563106796116506e-06,
+      "loss": 0.1363,
+      "step": 1285
+    },
+    {
+      "epoch": 5.947429231658001,
+      "grad_norm": 0.36234650015830994,
+      "learning_rate": 1.051779935275081e-06,
+      "loss": 0.127,
+      "step": 1290
+    },
+    {
+      "epoch": 5.97053726169844,
+      "grad_norm": 0.46252524852752686,
+      "learning_rate": 6.472491909385114e-07,
+      "loss": 0.1247,
+      "step": 1295
+    },
+    {
+      "epoch": 5.993645291738879,
+      "grad_norm": 0.40164050459861755,
+      "learning_rate": 2.4271844660194176e-07,
+      "loss": 0.1267,
+      "step": 1300
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 1302,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 6,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2.116928596049553e+19,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Qwen3-4B-Instruct-2507-sft-fusang-swa=2k_sink=0/checkpoint-1736/added_tokens.json

@@ -0,0 +1,28 @@
+{
+  "</think>": 151668,
+  "</tool_call>": 151658,
+  "</tool_response>": 151666,
+  "<think>": 151667,
+  "<tool_call>": 151657,
+  "<tool_response>": 151665,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

Qwen3-4B-Instruct-2507-sft-fusang-swa=2k_sink=0/checkpoint-1736/tokenizer_config.json

@@ -0,0 +1,239 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151665": {
+      "content": "<tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151666": {
+      "content": "</tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151667": {
+      "content": "<think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151668": {
+      "content": "</think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 1010000,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

Qwen3-4B-Instruct-2507-sft-fusang-swa=2k_sink=0/checkpoint-1736/trainer_state.json

@@ -0,0 +1,2463 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 8.0,
+  "eval_steps": 500,
+  "global_step": 1736,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.023108030040439053,
+      "grad_norm": 1.824995756149292,
+      "learning_rate": 4.5977011494252875e-06,
+      "loss": 0.6532,
+      "step": 5
+    },
+    {
+      "epoch": 0.046216060080878106,
+      "grad_norm": 4.508100986480713,
+      "learning_rate": 1.0344827586206897e-05,
+      "loss": 0.6156,
+      "step": 10
+    },
+    {
+      "epoch": 0.06932409012131716,
+      "grad_norm": 1.9596883058547974,
+      "learning_rate": 1.6091954022988507e-05,
+      "loss": 0.5458,
+      "step": 15
+    },
+    {
+      "epoch": 0.09243212016175621,
+      "grad_norm": 1.8338247537612915,
+      "learning_rate": 2.183908045977012e-05,
+      "loss": 0.4778,
+      "step": 20
+    },
+    {
+      "epoch": 0.11554015020219527,
+      "grad_norm": 0.9364601969718933,
+      "learning_rate": 2.7586206896551727e-05,
+      "loss": 0.441,
+      "step": 25
+    },
+    {
+      "epoch": 0.1386481802426343,
+      "grad_norm": 1.145851492881775,
+      "learning_rate": 3.3333333333333335e-05,
+      "loss": 0.4446,
+      "step": 30
+    },
+    {
+      "epoch": 0.16175621028307338,
+      "grad_norm": 0.8513086438179016,
+      "learning_rate": 3.908045977011495e-05,
+      "loss": 0.4353,
+      "step": 35
+    },
+    {
+      "epoch": 0.18486424032351242,
+      "grad_norm": 0.8944790959358215,
+      "learning_rate": 4.482758620689655e-05,
+      "loss": 0.3781,
+      "step": 40
+    },
+    {
+      "epoch": 0.20797227036395147,
+      "grad_norm": 1.0144641399383545,
+      "learning_rate": 5.057471264367817e-05,
+      "loss": 0.3698,
+      "step": 45
+    },
+    {
+      "epoch": 0.23108030040439054,
+      "grad_norm": 0.6831822991371155,
+      "learning_rate": 5.632183908045977e-05,
+      "loss": 0.3475,
+      "step": 50
+    },
+    {
+      "epoch": 0.2541883304448296,
+      "grad_norm": 0.6706651449203491,
+      "learning_rate": 6.206896551724138e-05,
+      "loss": 0.3445,
+      "step": 55
+    },
+    {
+      "epoch": 0.2772963604852686,
+      "grad_norm": 0.7026403546333313,
+      "learning_rate": 6.781609195402298e-05,
+      "loss": 0.3407,
+      "step": 60
+    },
+    {
+      "epoch": 0.30040439052570767,
+      "grad_norm": 1.1292873620986938,
+      "learning_rate": 7.35632183908046e-05,
+      "loss": 0.2975,
+      "step": 65
+    },
+    {
+      "epoch": 0.32351242056614676,
+      "grad_norm": 1.0919313430786133,
+      "learning_rate": 7.931034482758621e-05,
+      "loss": 0.3029,
+      "step": 70
+    },
+    {
+      "epoch": 0.3466204506065858,
+      "grad_norm": 0.7146428227424622,
+      "learning_rate": 8.505747126436782e-05,
+      "loss": 0.3017,
+      "step": 75
+    },
+    {
+      "epoch": 0.36972848064702485,
+      "grad_norm": 0.6972354650497437,
+      "learning_rate": 9.080459770114943e-05,
+      "loss": 0.3067,
+      "step": 80
+    },
+    {
+      "epoch": 0.3928365106874639,
+      "grad_norm": 0.7083359360694885,
+      "learning_rate": 9.655172413793105e-05,
+      "loss": 0.2968,
+      "step": 85
+    },
+    {
+      "epoch": 0.41594454072790293,
+      "grad_norm": 0.9471346735954285,
+      "learning_rate": 9.999963704071673e-05,
+      "loss": 0.2902,
+      "step": 90
+    },
+    {
+      "epoch": 0.439052570768342,
+      "grad_norm": 1.250731348991394,
+      "learning_rate": 9.999555380929755e-05,
+      "loss": 0.2776,
+      "step": 95
+    },
+    {
+      "epoch": 0.4621606008087811,
+      "grad_norm": 0.6727545261383057,
+      "learning_rate": 9.998693401909946e-05,
+      "loss": 0.276,
+      "step": 100
+    },
+    {
+      "epoch": 0.4852686308492201,
+      "grad_norm": 0.595573365688324,
+      "learning_rate": 9.997377845227576e-05,
+      "loss": 0.2881,
+      "step": 105
+    },
+    {
+      "epoch": 0.5083766608896592,
+      "grad_norm": 0.6969460844993591,
+      "learning_rate": 9.995608830255258e-05,
+      "loss": 0.288,
+      "step": 110
+    },
+    {
+      "epoch": 0.5314846909300982,
+      "grad_norm": 0.7695512175559998,
+      "learning_rate": 9.993386517512082e-05,
+      "loss": 0.2622,
+      "step": 115
+    },
+    {
+      "epoch": 0.5545927209705372,
+      "grad_norm": 0.8070512413978577,
+      "learning_rate": 9.99071110864902e-05,
+      "loss": 0.2771,
+      "step": 120
+    },
+    {
+      "epoch": 0.5777007510109763,
+      "grad_norm": 0.8525498509407043,
+      "learning_rate": 9.987582846430656e-05,
+      "loss": 0.2691,
+      "step": 125
+    },
+    {
+      "epoch": 0.6008087810514153,
+      "grad_norm": 0.6688218712806702,
+      "learning_rate": 9.984002014713138e-05,
+      "loss": 0.274,
+      "step": 130
+    },
+    {
+      "epoch": 0.6239168110918544,
+      "grad_norm": 0.6929794549942017,
+      "learning_rate": 9.97996893841843e-05,
+      "loss": 0.2729,
+      "step": 135
+    },
+    {
+      "epoch": 0.6470248411322935,
+      "grad_norm": 0.6344878077507019,
+      "learning_rate": 9.975483983504829e-05,
+      "loss": 0.2721,
+      "step": 140
+    },
+    {
+      "epoch": 0.6701328711727326,
+      "grad_norm": 0.714883029460907,
+      "learning_rate": 9.970547556933754e-05,
+      "loss": 0.2652,
+      "step": 145
+    },
+    {
+      "epoch": 0.6932409012131716,
+      "grad_norm": 0.5451264381408691,
+      "learning_rate": 9.965160106632823e-05,
+      "loss": 0.2582,
+      "step": 150
+    },
+    {
+      "epoch": 0.7163489312536107,
+      "grad_norm": 0.5924599170684814,
+      "learning_rate": 9.959322121455208e-05,
+      "loss": 0.2655,
+      "step": 155
+    },
+    {
+      "epoch": 0.7394569612940497,
+      "grad_norm": 0.6610844135284424,
+      "learning_rate": 9.95303413113527e-05,
+      "loss": 0.2623,
+      "step": 160
+    },
+    {
+      "epoch": 0.7625649913344887,
+      "grad_norm": 1.5605195760726929,
+      "learning_rate": 9.946296706240501e-05,
+      "loss": 0.2547,
+      "step": 165
+    },
+    {
+      "epoch": 0.7856730213749278,
+      "grad_norm": 0.6948598623275757,
+      "learning_rate": 9.939110458119745e-05,
+      "loss": 0.2515,
+      "step": 170
+    },
+    {
+      "epoch": 0.8087810514153668,
+      "grad_norm": 0.43471047282218933,
+      "learning_rate": 9.931476038847729e-05,
+      "loss": 0.2519,
+      "step": 175
+    },
+    {
+      "epoch": 0.8318890814558059,
+      "grad_norm": 0.6591277122497559,
+      "learning_rate": 9.923394141165888e-05,
+      "loss": 0.2714,
+      "step": 180
+    },
+    {
+      "epoch": 0.8549971114962449,
+      "grad_norm": 0.548107385635376,
+      "learning_rate": 9.91486549841951e-05,
+      "loss": 0.2647,
+      "step": 185
+    },
+    {
+      "epoch": 0.878105141536684,
+      "grad_norm": 0.6994490623474121,
+      "learning_rate": 9.905890884491195e-05,
+      "loss": 0.249,
+      "step": 190
+    },
+    {
+      "epoch": 0.901213171577123,
+      "grad_norm": 1.0918959379196167,
+      "learning_rate": 9.896471113730632e-05,
+      "loss": 0.2542,
+      "step": 195
+    },
+    {
+      "epoch": 0.9243212016175621,
+      "grad_norm": 0.5028172731399536,
+      "learning_rate": 9.886607040880702e-05,
+      "loss": 0.2509,
+      "step": 200
+    },
+    {
+      "epoch": 0.9474292316580012,
+      "grad_norm": 0.739396870136261,
+      "learning_rate": 9.876299560999922e-05,
+      "loss": 0.2544,
+      "step": 205
+    },
+    {
+      "epoch": 0.9705372616984402,
+      "grad_norm": 0.5704207420349121,
+      "learning_rate": 9.86554960938123e-05,
+      "loss": 0.2599,
+      "step": 210
+    },
+    {
+      "epoch": 0.9936452917388793,
+      "grad_norm": 0.8608630299568176,
+      "learning_rate": 9.854358161467109e-05,
+      "loss": 0.2558,
+      "step": 215
+    },
+    {
+      "epoch": 1.0138648180242635,
+      "grad_norm": 1.7093055248260498,
+      "learning_rate": 9.842726232761095e-05,
+      "loss": 0.2536,
+      "step": 220
+    },
+    {
+      "epoch": 1.0369728480647025,
+      "grad_norm": 1.0725808143615723,
+      "learning_rate": 9.830654878735604e-05,
+      "loss": 0.2413,
+      "step": 225
+    },
+    {
+      "epoch": 1.0600808781051416,
+      "grad_norm": 0.616876482963562,
+      "learning_rate": 9.818145194736183e-05,
+      "loss": 0.2348,
+      "step": 230
+    },
+    {
+      "epoch": 1.0831889081455806,
+      "grad_norm": 0.5362911820411682,
+      "learning_rate": 9.805198315882105e-05,
+      "loss": 0.2381,
+      "step": 235
+    },
+    {
+      "epoch": 1.1062969381860197,
+      "grad_norm": 0.6959365010261536,
+      "learning_rate": 9.791815416963376e-05,
+      "loss": 0.2579,
+      "step": 240
+    },
+    {
+      "epoch": 1.1294049682264586,
+      "grad_norm": 0.582306444644928,
+      "learning_rate": 9.777997712334133e-05,
+      "loss": 0.2307,
+      "step": 245
+    },
+    {
+      "epoch": 1.1525129982668978,
+      "grad_norm": 0.8091015815734863,
+      "learning_rate": 9.763746455802452e-05,
+      "loss": 0.2429,
+      "step": 250
+    },
+    {
+      "epoch": 1.1756210283073367,
+      "grad_norm": 0.7487279772758484,
+      "learning_rate": 9.749062940516585e-05,
+      "loss": 0.2253,
+      "step": 255
+    },
+    {
+      "epoch": 1.1987290583477759,
+      "grad_norm": 0.46290847659111023,
+      "learning_rate": 9.733948498847616e-05,
+      "loss": 0.2377,
+      "step": 260
+    },
+    {
+      "epoch": 1.2218370883882148,
+      "grad_norm": 0.702442467212677,
+      "learning_rate": 9.71840450226856e-05,
+      "loss": 0.2394,
+      "step": 265
+    },
+    {
+      "epoch": 1.244945118428654,
+      "grad_norm": 0.6691907048225403,
+      "learning_rate": 9.702432361229926e-05,
+      "loss": 0.2302,
+      "step": 270
+    },
+    {
+      "epoch": 1.2680531484690931,
+      "grad_norm": 0.6166519522666931,
+      "learning_rate": 9.686033525031719e-05,
+      "loss": 0.2357,
+      "step": 275
+    },
+    {
+      "epoch": 1.291161178509532,
+      "grad_norm": 0.7439646124839783,
+      "learning_rate": 9.66920948169195e-05,
+      "loss": 0.2313,
+      "step": 280
+    },
+    {
+      "epoch": 1.314269208549971,
+      "grad_norm": 0.4383182227611542,
+      "learning_rate": 9.651961757811598e-05,
+      "loss": 0.231,
+      "step": 285
+    },
+    {
+      "epoch": 1.3373772385904101,
+      "grad_norm": 0.5435488820075989,
+      "learning_rate": 9.634291918436098e-05,
+      "loss": 0.2374,
+      "step": 290
+    },
+    {
+      "epoch": 1.3604852686308493,
+      "grad_norm": 1.2654523849487305,
+      "learning_rate": 9.616201566913325e-05,
+      "loss": 0.2369,
+      "step": 295
+    },
+    {
+      "epoch": 1.3835932986712882,
+      "grad_norm": 0.5717748403549194,
+      "learning_rate": 9.597692344748106e-05,
+      "loss": 0.2357,
+      "step": 300
+    },
+    {
+      "epoch": 1.4067013287117274,
+      "grad_norm": 0.5190787315368652,
+      "learning_rate": 9.578765931453278e-05,
+      "loss": 0.2217,
+      "step": 305
+    },
+    {
+      "epoch": 1.4298093587521663,
+      "grad_norm": 0.5561380386352539,
+      "learning_rate": 9.55942404439728e-05,
+      "loss": 0.2416,
+      "step": 310
+    },
+    {
+      "epoch": 1.4529173887926055,
+      "grad_norm": 0.9981644749641418,
+      "learning_rate": 9.539668438648335e-05,
+      "loss": 0.2395,
+      "step": 315
+    },
+    {
+      "epoch": 1.4760254188330446,
+      "grad_norm": 0.5633391737937927,
+      "learning_rate": 9.519500906815179e-05,
+      "loss": 0.2257,
+      "step": 320
+    },
+    {
+      "epoch": 1.4991334488734835,
+      "grad_norm": 0.6023544669151306,
+      "learning_rate": 9.498923278884414e-05,
+      "loss": 0.2331,
+      "step": 325
+    },
+    {
+      "epoch": 1.5222414789139225,
+      "grad_norm": 0.5386343002319336,
+      "learning_rate": 9.477937422054451e-05,
+      "loss": 0.2295,
+      "step": 330
+    },
+    {
+      "epoch": 1.5453495089543616,
+      "grad_norm": 0.5330613255500793,
+      "learning_rate": 9.456545240566084e-05,
+      "loss": 0.2353,
+      "step": 335
+    },
+    {
+      "epoch": 1.5684575389948008,
+      "grad_norm": 0.5590543150901794,
+      "learning_rate": 9.434748675529697e-05,
+      "loss": 0.2448,
+      "step": 340
+    },
+    {
+      "epoch": 1.5915655690352397,
+      "grad_norm": 0.5556681156158447,
+      "learning_rate": 9.412549704749134e-05,
+      "loss": 0.2377,
+      "step": 345
+    },
+    {
+      "epoch": 1.6146735990756786,
+      "grad_norm": 0.5753155946731567,
+      "learning_rate": 9.389950342542231e-05,
+      "loss": 0.2282,
+      "step": 350
+    },
+    {
+      "epoch": 1.6377816291161178,
+      "grad_norm": 0.5049495100975037,
+      "learning_rate": 9.36695263955804e-05,
+      "loss": 0.2283,
+      "step": 355
+    },
+    {
+      "epoch": 1.660889659156557,
+      "grad_norm": 0.5120885968208313,
+      "learning_rate": 9.343558682590756e-05,
+      "loss": 0.2346,
+      "step": 360
+    },
+    {
+      "epoch": 1.683997689196996,
+      "grad_norm": 0.6427890062332153,
+      "learning_rate": 9.319770594390356e-05,
+      "loss": 0.2394,
+      "step": 365
+    },
+    {
+      "epoch": 1.707105719237435,
+      "grad_norm": 0.5295055508613586,
+      "learning_rate": 9.295590533469991e-05,
+      "loss": 0.2198,
+      "step": 370
+    },
+    {
+      "epoch": 1.730213749277874,
+      "grad_norm": 0.5124744176864624,
+      "learning_rate": 9.271020693910122e-05,
+      "loss": 0.2289,
+      "step": 375
+    },
+    {
+      "epoch": 1.7533217793183131,
+      "grad_norm": 0.5832476019859314,
+      "learning_rate": 9.246063305159425e-05,
+      "loss": 0.2237,
+      "step": 380
+    },
+    {
+      "epoch": 1.7764298093587523,
+      "grad_norm": 0.4641379117965698,
+      "learning_rate": 9.2207206318325e-05,
+      "loss": 0.2424,
+      "step": 385
+    },
+    {
+      "epoch": 1.7995378393991912,
+      "grad_norm": 0.5052978992462158,
+      "learning_rate": 9.194994973504377e-05,
+      "loss": 0.2454,
+      "step": 390
+    },
+    {
+      "epoch": 1.8226458694396301,
+      "grad_norm": 0.5559684634208679,
+      "learning_rate": 9.168888664501853e-05,
+      "loss": 0.235,
+      "step": 395
+    },
+    {
+      "epoch": 1.8457538994800693,
+      "grad_norm": 0.5935572385787964,
+      "learning_rate": 9.142404073691686e-05,
+      "loss": 0.2329,
+      "step": 400
+    },
+    {
+      "epoch": 1.8688619295205084,
+      "grad_norm": 0.4894205331802368,
+      "learning_rate": 9.115543604265636e-05,
+      "loss": 0.2266,
+      "step": 405
+    },
+    {
+      "epoch": 1.8919699595609474,
+      "grad_norm": 0.5184963345527649,
+      "learning_rate": 9.088309693522404e-05,
+      "loss": 0.2298,
+      "step": 410
+    },
+    {
+      "epoch": 1.9150779896013865,
+      "grad_norm": 0.5215513706207275,
+      "learning_rate": 9.060704812646472e-05,
+      "loss": 0.2237,
+      "step": 415
+    },
+    {
+      "epoch": 1.9381860196418255,
+      "grad_norm": 0.5198848247528076,
+      "learning_rate": 9.032731466483877e-05,
+      "loss": 0.229,
+      "step": 420
+    },
+    {
+      "epoch": 1.9612940496822646,
+      "grad_norm": 0.6438161730766296,
+      "learning_rate": 9.004392193314913e-05,
+      "loss": 0.2307,
+      "step": 425
+    },
+    {
+      "epoch": 1.9844020797227038,
+      "grad_norm": 0.7170857191085815,
+      "learning_rate": 8.975689564623818e-05,
+      "loss": 0.2336,
+      "step": 430
+    },
+    {
+      "epoch": 2.004621606008088,
+      "grad_norm": 0.7200104594230652,
+      "learning_rate": 8.94662618486543e-05,
+      "loss": 0.2268,
+      "step": 435
+    },
+    {
+      "epoch": 2.027729636048527,
+      "grad_norm": 0.520868182182312,
+      "learning_rate": 8.917204691228872e-05,
+      "loss": 0.2016,
+      "step": 440
+    },
+    {
+      "epoch": 2.050837666088966,
+      "grad_norm": 0.7679985761642456,
+      "learning_rate": 8.887427753398248e-05,
+      "loss": 0.2067,
+      "step": 445
+    },
+    {
+      "epoch": 2.073945696129405,
+      "grad_norm": 0.6079690456390381,
+      "learning_rate": 8.857298073310396e-05,
+      "loss": 0.2107,
+      "step": 450
+    },
+    {
+      "epoch": 2.097053726169844,
+      "grad_norm": 0.5361660718917847,
+      "learning_rate": 8.826818384909726e-05,
+      "loss": 0.2091,
+      "step": 455
+    },
+    {
+      "epoch": 2.1201617562102832,
+      "grad_norm": 0.6037284135818481,
+      "learning_rate": 8.795991453900132e-05,
+      "loss": 0.2095,
+      "step": 460
+    },
+    {
+      "epoch": 2.143269786250722,
+      "grad_norm": 0.4307143986225128,
+      "learning_rate": 8.764820077494047e-05,
+      "loss": 0.2076,
+      "step": 465
+    },
+    {
+      "epoch": 2.166377816291161,
+      "grad_norm": 0.43318474292755127,
+      "learning_rate": 8.73330708415861e-05,
+      "loss": 0.2189,
+      "step": 470
+    },
+    {
+      "epoch": 2.1894858463316003,
+      "grad_norm": 0.5127202272415161,
+      "learning_rate": 8.70145533335903e-05,
+      "loss": 0.2183,
+      "step": 475
+    },
+    {
+      "epoch": 2.2125938763720394,
+      "grad_norm": 0.7460241317749023,
+      "learning_rate": 8.669267715299107e-05,
+      "loss": 0.2075,
+      "step": 480
+    },
+    {
+      "epoch": 2.235701906412478,
+      "grad_norm": 0.649926483631134,
+      "learning_rate": 8.636747150658986e-05,
+      "loss": 0.2133,
+      "step": 485
+    },
+    {
+      "epoch": 2.2588099364529173,
+      "grad_norm": 0.6260571479797363,
+      "learning_rate": 8.603896590330133e-05,
+      "loss": 0.2108,
+      "step": 490
+    },
+    {
+      "epoch": 2.2819179664933564,
+      "grad_norm": 0.5095546245574951,
+      "learning_rate": 8.570719015147569e-05,
+      "loss": 0.2167,
+      "step": 495
+    },
+    {
+      "epoch": 2.3050259965337956,
+      "grad_norm": 0.7540727853775024,
+      "learning_rate": 8.537217435619403e-05,
+      "loss": 0.2074,
+      "step": 500
+    },
+    {
+      "epoch": 2.3281340265742347,
+      "grad_norm": 1.7953331470489502,
+      "learning_rate": 8.503394891653645e-05,
+      "loss": 0.2064,
+      "step": 505
+    },
+    {
+      "epoch": 2.3512420566146734,
+      "grad_norm": 0.6022052764892578,
+      "learning_rate": 8.469254452282384e-05,
+      "loss": 0.2113,
+      "step": 510
+    },
+    {
+      "epoch": 2.3743500866551126,
+      "grad_norm": 0.5126685500144958,
+      "learning_rate": 8.434799215383291e-05,
+      "loss": 0.2233,
+      "step": 515
+    },
+    {
+      "epoch": 2.3974581166955518,
+      "grad_norm": 0.49502256512641907,
+      "learning_rate": 8.400032307398533e-05,
+      "loss": 0.2136,
+      "step": 520
+    },
+    {
+      "epoch": 2.420566146735991,
+      "grad_norm": 0.7737590074539185,
+      "learning_rate": 8.364956883051069e-05,
+      "loss": 0.2158,
+      "step": 525
+    },
+    {
+      "epoch": 2.4436741767764296,
+      "grad_norm": 0.5639827847480774,
+      "learning_rate": 8.329576125058406e-05,
+      "loss": 0.2135,
+      "step": 530
+    },
+    {
+      "epoch": 2.4667822068168688,
+      "grad_norm": 1.0111901760101318,
+      "learning_rate": 8.293893243843793e-05,
+      "loss": 0.2105,
+      "step": 535
+    },
+    {
+      "epoch": 2.489890236857308,
+      "grad_norm": 0.6146679520606995,
+      "learning_rate": 8.257911477244907e-05,
+      "loss": 0.2087,
+      "step": 540
+    },
+    {
+      "epoch": 2.512998266897747,
+      "grad_norm": 0.603611171245575,
+      "learning_rate": 8.22163409022007e-05,
+      "loss": 0.2255,
+      "step": 545
+    },
+    {
+      "epoch": 2.5361062969381862,
+      "grad_norm": 0.5318015813827515,
+      "learning_rate": 8.185064374551966e-05,
+      "loss": 0.2208,
+      "step": 550
+    },
+    {
+      "epoch": 2.559214326978625,
+      "grad_norm": 0.5801270008087158,
+      "learning_rate": 8.148205648548968e-05,
+      "loss": 0.2136,
+      "step": 555
+    },
+    {
+      "epoch": 2.582322357019064,
+      "grad_norm": 0.8735573887825012,
+      "learning_rate": 8.111061256744024e-05,
+      "loss": 0.2163,
+      "step": 560
+    },
+    {
+      "epoch": 2.6054303870595032,
+      "grad_norm": 0.8433005809783936,
+      "learning_rate": 8.073634569591182e-05,
+      "loss": 0.2116,
+      "step": 565
+    },
+    {
+      "epoch": 2.628538417099942,
+      "grad_norm": 0.7705909013748169,
+      "learning_rate": 8.035928983159757e-05,
+      "loss": 0.2129,
+      "step": 570
+    },
+    {
+      "epoch": 2.651646447140381,
+      "grad_norm": 0.5460626482963562,
+      "learning_rate": 7.997947918826174e-05,
+      "loss": 0.2216,
+      "step": 575
+    },
+    {
+      "epoch": 2.6747544771808203,
+      "grad_norm": 0.455725759267807,
+      "learning_rate": 7.959694822963515e-05,
+      "loss": 0.2084,
+      "step": 580
+    },
+    {
+      "epoch": 2.6978625072212594,
+      "grad_norm": 0.5386655926704407,
+      "learning_rate": 7.921173166628795e-05,
+      "loss": 0.2086,
+      "step": 585
+    },
+    {
+      "epoch": 2.7209705372616986,
+      "grad_norm": 0.49504733085632324,
+      "learning_rate": 7.882386445248004e-05,
+      "loss": 0.2032,
+      "step": 590
+    },
+    {
+      "epoch": 2.7440785673021377,
+      "grad_norm": 0.5183764100074768,
+      "learning_rate": 7.84333817829894e-05,
+      "loss": 0.216,
+      "step": 595
+    },
+    {
+      "epoch": 2.7671865973425764,
+      "grad_norm": 0.5233456492424011,
+      "learning_rate": 7.804031908991838e-05,
+      "loss": 0.2129,
+      "step": 600
+    },
+    {
+      "epoch": 2.7902946273830156,
+      "grad_norm": 0.6231764554977417,
+      "learning_rate": 7.76447120394788e-05,
+      "loss": 0.2119,
+      "step": 605
+    },
+    {
+      "epoch": 2.8134026574234547,
+      "grad_norm": 0.5794748067855835,
+      "learning_rate": 7.724659652875554e-05,
+      "loss": 0.2115,
+      "step": 610
+    },
+    {
+      "epoch": 2.8365106874638935,
+      "grad_norm": 0.48975124955177307,
+      "learning_rate": 7.68460086824492e-05,
+      "loss": 0.2154,
+      "step": 615
+    },
+    {
+      "epoch": 2.8596187175043326,
+      "grad_norm": 0.5089766979217529,
+      "learning_rate": 7.644298484959831e-05,
+      "loss": 0.2166,
+      "step": 620
+    },
+    {
+      "epoch": 2.8827267475447718,
+      "grad_norm": 0.5563911199569702,
+      "learning_rate": 7.603756160028099e-05,
+      "loss": 0.2086,
+      "step": 625
+    },
+    {
+      "epoch": 2.905834777585211,
+      "grad_norm": 1.035365104675293,
+      "learning_rate": 7.562977572229654e-05,
+      "loss": 0.2069,
+      "step": 630
+    },
+    {
+      "epoch": 2.92894280762565,
+      "grad_norm": 0.86057448387146,
+      "learning_rate": 7.521966421782743e-05,
+      "loss": 0.2093,
+      "step": 635
+    },
+    {
+      "epoch": 2.9520508376660892,
+      "grad_norm": 0.42633533477783203,
+      "learning_rate": 7.480726430008174e-05,
+      "loss": 0.207,
+      "step": 640
+    },
+    {
+      "epoch": 2.975158867706528,
+      "grad_norm": 0.6084079742431641,
+      "learning_rate": 7.439261338991639e-05,
+      "loss": 0.2139,
+      "step": 645
+    },
+    {
+      "epoch": 2.998266897746967,
+      "grad_norm": 0.4970872402191162,
+      "learning_rate": 7.397574911244168e-05,
+      "loss": 0.2097,
+      "step": 650
+    },
+    {
+      "epoch": 3.0184864240323512,
+      "grad_norm": 0.49431514739990234,
+      "learning_rate": 7.355670929360725e-05,
+      "loss": 0.2114,
+      "step": 655
+    },
+    {
+      "epoch": 3.0415944540727904,
+      "grad_norm": 0.5486884117126465,
+      "learning_rate": 7.313553195676959e-05,
+      "loss": 0.1999,
+      "step": 660
+    },
+    {
+      "epoch": 3.0647024841132295,
+      "grad_norm": 0.5666886568069458,
+      "learning_rate": 7.271225531924204e-05,
+      "loss": 0.2014,
+      "step": 665
+    },
+    {
+      "epoch": 3.0878105141536683,
+      "grad_norm": 0.4839744567871094,
+      "learning_rate": 7.228691778882693e-05,
+      "loss": 0.1886,
+      "step": 670
+    },
+    {
+      "epoch": 3.1109185441941074,
+      "grad_norm": 0.4740411639213562,
+      "learning_rate": 7.185955796033035e-05,
+      "loss": 0.1995,
+      "step": 675
+    },
+    {
+      "epoch": 3.1340265742345466,
+      "grad_norm": 0.5649032592773438,
+      "learning_rate": 7.143021461206029e-05,
+      "loss": 0.1937,
+      "step": 680
+    },
+    {
+      "epoch": 3.1571346042749857,
+      "grad_norm": 0.6299188733100891,
+      "learning_rate": 7.099892670230777e-05,
+      "loss": 0.1924,
+      "step": 685
+    },
+    {
+      "epoch": 3.1802426343154244,
+      "grad_norm": 0.8647441864013672,
+      "learning_rate": 7.056573336581187e-05,
+      "loss": 0.1875,
+      "step": 690
+    },
+    {
+      "epoch": 3.2033506643558636,
+      "grad_norm": 0.4554251432418823,
+      "learning_rate": 7.013067391020864e-05,
+      "loss": 0.1917,
+      "step": 695
+    },
+    {
+      "epoch": 3.2264586943963027,
+      "grad_norm": 0.522324800491333,
+      "learning_rate": 6.969378781246436e-05,
+      "loss": 0.2013,
+      "step": 700
+    },
+    {
+      "epoch": 3.249566724436742,
+      "grad_norm": 0.47249627113342285,
+      "learning_rate": 6.925511471529348e-05,
+      "loss": 0.1854,
+      "step": 705
+    },
+    {
+      "epoch": 3.2726747544771806,
+      "grad_norm": 0.5154001712799072,
+      "learning_rate": 6.881469442356141e-05,
+      "loss": 0.1957,
+      "step": 710
+    },
+    {
+      "epoch": 3.2957827845176197,
+      "grad_norm": 0.5822983980178833,
+      "learning_rate": 6.837256690067266e-05,
+      "loss": 0.1931,
+      "step": 715
+    },
+    {
+      "epoch": 3.318890814558059,
+      "grad_norm": 0.4296785891056061,
+      "learning_rate": 6.792877226494457e-05,
+      "loss": 0.1955,
+      "step": 720
+    },
+    {
+      "epoch": 3.341998844598498,
+      "grad_norm": 0.5408909320831299,
+      "learning_rate": 6.748335078596713e-05,
+      "loss": 0.1972,
+      "step": 725
+    },
+    {
+      "epoch": 3.365106874638937,
+      "grad_norm": 0.4329081177711487,
+      "learning_rate": 6.703634288094877e-05,
+      "loss": 0.1985,
+      "step": 730
+    },
+    {
+      "epoch": 3.388214904679376,
+      "grad_norm": 1.0171910524368286,
+      "learning_rate": 6.658778911104909e-05,
+      "loss": 0.1943,
+      "step": 735
+    },
+    {
+      "epoch": 3.411322934719815,
+      "grad_norm": 0.6055728793144226,
+      "learning_rate": 6.613773017769826e-05,
+      "loss": 0.1918,
+      "step": 740
+    },
+    {
+      "epoch": 3.4344309647602542,
+      "grad_norm": 0.44774141907691956,
+      "learning_rate": 6.568620691890383e-05,
+      "loss": 0.1897,
+      "step": 745
+    },
+    {
+      "epoch": 3.4575389948006934,
+      "grad_norm": 0.5070408582687378,
+      "learning_rate": 6.523326030554518e-05,
+      "loss": 0.1908,
+      "step": 750
+    },
+    {
+      "epoch": 3.480647024841132,
+      "grad_norm": 0.5304681062698364,
+      "learning_rate": 6.477893143765579e-05,
+      "loss": 0.1942,
+      "step": 755
+    },
+    {
+      "epoch": 3.5037550548815712,
+      "grad_norm": 1.922534704208374,
+      "learning_rate": 6.432326154069385e-05,
+      "loss": 0.2047,
+      "step": 760
+    },
+    {
+      "epoch": 3.5268630849220104,
+      "grad_norm": 0.5374747514724731,
+      "learning_rate": 6.386629196180155e-05,
+      "loss": 0.1916,
+      "step": 765
+    },
+    {
+      "epoch": 3.5499711149624495,
+      "grad_norm": 0.6878522038459778,
+      "learning_rate": 6.340806416605321e-05,
+      "loss": 0.1909,
+      "step": 770
+    },
+    {
+      "epoch": 3.5730791450028887,
+      "grad_norm": 0.49280133843421936,
+      "learning_rate": 6.294861973269278e-05,
+      "loss": 0.2009,
+      "step": 775
+    },
+    {
+      "epoch": 3.5961871750433274,
+      "grad_norm": 1.1669367551803589,
+      "learning_rate": 6.248800035136106e-05,
+      "loss": 0.19,
+      "step": 780
+    },
+    {
+      "epoch": 3.6192952050837666,
+      "grad_norm": 0.5372188687324524,
+      "learning_rate": 6.202624781831268e-05,
+      "loss": 0.2014,
+      "step": 785
+    },
+    {
+      "epoch": 3.6424032351242057,
+      "grad_norm": 5.554832935333252,
+      "learning_rate": 6.156340403262361e-05,
+      "loss": 0.1951,
+      "step": 790
+    },
+    {
+      "epoch": 3.665511265164645,
+      "grad_norm": 0.6088634729385376,
+      "learning_rate": 6.109951099238925e-05,
+      "loss": 0.1924,
+      "step": 795
+    },
+    {
+      "epoch": 3.6886192952050836,
+      "grad_norm": 0.4436858594417572,
+      "learning_rate": 6.0634610790913604e-05,
+      "loss": 0.1941,
+      "step": 800
+    },
+    {
+      "epoch": 3.7117273252455227,
+      "grad_norm": 0.6255747079849243,
+      "learning_rate": 6.0168745612889685e-05,
+      "loss": 0.187,
+      "step": 805
+    },
+    {
+      "epoch": 3.734835355285962,
+      "grad_norm": 0.7627413272857666,
+      "learning_rate": 5.9701957730571786e-05,
+      "loss": 0.1978,
+      "step": 810
+    },
+    {
+      "epoch": 3.757943385326401,
+      "grad_norm": 0.5394768714904785,
+      "learning_rate": 5.923428949993964e-05,
+      "loss": 0.1966,
+      "step": 815
+    },
+    {
+      "epoch": 3.78105141536684,
+      "grad_norm": 0.43580150604248047,
+      "learning_rate": 5.876578335685518e-05,
+      "loss": 0.191,
+      "step": 820
+    },
+    {
+      "epoch": 3.804159445407279,
+      "grad_norm": 0.4186820685863495,
+      "learning_rate": 5.829648181321183e-05,
+      "loss": 0.1902,
+      "step": 825
+    },
+    {
+      "epoch": 3.827267475447718,
+      "grad_norm": 0.4860546290874481,
+      "learning_rate": 5.782642745307712e-05,
+      "loss": 0.2023,
+      "step": 830
+    },
+    {
+      "epoch": 3.850375505488157,
+      "grad_norm": 0.45939549803733826,
+      "learning_rate": 5.735566292882849e-05,
+      "loss": 0.2016,
+      "step": 835
+    },
+    {
+      "epoch": 3.873483535528596,
+      "grad_norm": 0.5214559435844421,
+      "learning_rate": 5.6884230957283225e-05,
+      "loss": 0.1965,
+      "step": 840
+    },
+    {
+      "epoch": 3.896591565569035,
+      "grad_norm": 0.4323074221611023,
+      "learning_rate": 5.641217431582223e-05,
+      "loss": 0.1938,
+      "step": 845
+    },
+    {
+      "epoch": 3.9196995956094742,
+      "grad_norm": 0.47958096861839294,
+      "learning_rate": 5.593953583850847e-05,
+      "loss": 0.1979,
+      "step": 850
+    },
+    {
+      "epoch": 3.9428076256499134,
+      "grad_norm": 0.7347373366355896,
+      "learning_rate": 5.5466358412200317e-05,
+      "loss": 0.2057,
+      "step": 855
+    },
+    {
+      "epoch": 3.9659156556903525,
+      "grad_norm": 0.6283802390098572,
+      "learning_rate": 5.49926849726599e-05,
+      "loss": 0.1937,
+      "step": 860
+    },
+    {
+      "epoch": 3.9890236857307917,
+      "grad_norm": 0.5379176139831543,
+      "learning_rate": 5.451855850065724e-05,
+      "loss": 0.1852,
+      "step": 865
+    },
+    {
+      "epoch": 4.009243212016176,
+      "grad_norm": 0.5923841595649719,
+      "learning_rate": 5.4044022018070214e-05,
+      "loss": 0.1951,
+      "step": 870
+    },
+    {
+      "epoch": 4.032351242056615,
+      "grad_norm": 0.5223307013511658,
+      "learning_rate": 5.356911858398068e-05,
+      "loss": 0.1754,
+      "step": 875
+    },
+    {
+      "epoch": 4.055459272097054,
+      "grad_norm": 0.44683995842933655,
+      "learning_rate": 5.309389129076747e-05,
+      "loss": 0.1832,
+      "step": 880
+    },
+    {
+      "epoch": 4.078567302137492,
+      "grad_norm": 0.5101492404937744,
+      "learning_rate": 5.2618383260196105e-05,
+      "loss": 0.1908,
+      "step": 885
+    },
+    {
+      "epoch": 4.101675332177932,
+      "grad_norm": 0.49731770157814026,
+      "learning_rate": 5.2142637639506e-05,
+      "loss": 0.1838,
+      "step": 890
+    },
+    {
+      "epoch": 4.124783362218371,
+      "grad_norm": 0.792204737663269,
+      "learning_rate": 5.1666697597495336e-05,
+      "loss": 0.1832,
+      "step": 895
+    },
+    {
+      "epoch": 4.14789139225881,
+      "grad_norm": 0.5713189840316772,
+      "learning_rate": 5.119060632060394e-05,
+      "loss": 0.1802,
+      "step": 900
+    },
+    {
+      "epoch": 4.170999422299249,
+      "grad_norm": 0.40742775797843933,
+      "learning_rate": 5.071440700899458e-05,
+      "loss": 0.1705,
+      "step": 905
+    },
+    {
+      "epoch": 4.194107452339688,
+      "grad_norm": 0.46602964401245117,
+      "learning_rate": 5.023814287263298e-05,
+      "loss": 0.1839,
+      "step": 910
+    },
+    {
+      "epoch": 4.217215482380127,
+      "grad_norm": 0.5177744030952454,
+      "learning_rate": 4.9761857127367034e-05,
+      "loss": 0.1871,
+      "step": 915
+    },
+    {
+      "epoch": 4.2403235124205665,
+      "grad_norm": 0.5244989395141602,
+      "learning_rate": 4.9285592991005434e-05,
+      "loss": 0.1825,
+      "step": 920
+    },
+    {
+      "epoch": 4.263431542461005,
+      "grad_norm": 0.5535411834716797,
+      "learning_rate": 4.880939367939607e-05,
+      "loss": 0.1748,
+      "step": 925
+    },
+    {
+      "epoch": 4.286539572501444,
+      "grad_norm": 0.6430193781852722,
+      "learning_rate": 4.833330240250468e-05,
+      "loss": 0.1805,
+      "step": 930
+    },
+    {
+      "epoch": 4.309647602541883,
+      "grad_norm": 0.4868832230567932,
+      "learning_rate": 4.785736236049402e-05,
+      "loss": 0.1744,
+      "step": 935
+    },
+    {
+      "epoch": 4.332755632582322,
+      "grad_norm": 0.6762482523918152,
+      "learning_rate": 4.7381616739803907e-05,
+      "loss": 0.1835,
+      "step": 940
+    },
+    {
+      "epoch": 4.355863662622761,
+      "grad_norm": 0.5419865250587463,
+      "learning_rate": 4.690610870923253e-05,
+      "loss": 0.1848,
+      "step": 945
+    },
+    {
+      "epoch": 4.3789716926632005,
+      "grad_norm": 0.6376410722732544,
+      "learning_rate": 4.6430881416019315e-05,
+      "loss": 0.1852,
+      "step": 950
+    },
+    {
+      "epoch": 4.40207972270364,
+      "grad_norm": 0.46862906217575073,
+      "learning_rate": 4.59559779819298e-05,
+      "loss": 0.1775,
+      "step": 955
+    },
+    {
+      "epoch": 4.425187752744079,
+      "grad_norm": 0.6234151721000671,
+      "learning_rate": 4.548144149934276e-05,
+      "loss": 0.1818,
+      "step": 960
+    },
+    {
+      "epoch": 4.448295782784518,
+      "grad_norm": 0.4400373101234436,
+      "learning_rate": 4.500731502734011e-05,
+      "loss": 0.1874,
+      "step": 965
+    },
+    {
+      "epoch": 4.471403812824956,
+      "grad_norm": 0.47349095344543457,
+      "learning_rate": 4.4533641587799695e-05,
+      "loss": 0.1742,
+      "step": 970
+    },
+    {
+      "epoch": 4.494511842865395,
+      "grad_norm": 0.55287104845047,
+      "learning_rate": 4.406046416149154e-05,
+      "loss": 0.1849,
+      "step": 975
+    },
+    {
+      "epoch": 4.5176198729058346,
+      "grad_norm": 0.421122670173645,
+      "learning_rate": 4.3587825684177797e-05,
+      "loss": 0.18,
+      "step": 980
+    },
+    {
+      "epoch": 4.540727902946274,
+      "grad_norm": 0.46700942516326904,
+      "learning_rate": 4.311576904271679e-05,
+      "loss": 0.1862,
+      "step": 985
+    },
+    {
+      "epoch": 4.563835932986713,
+      "grad_norm": 0.5416756868362427,
+      "learning_rate": 4.264433707117151e-05,
+      "loss": 0.1779,
+      "step": 990
+    },
+    {
+      "epoch": 4.586943963027152,
+      "grad_norm": 0.5200238227844238,
+      "learning_rate": 4.217357254692289e-05,
+      "loss": 0.183,
+      "step": 995
+    },
+    {
+      "epoch": 4.610051993067591,
+      "grad_norm": 0.6711512207984924,
+      "learning_rate": 4.170351818678817e-05,
+      "loss": 0.1794,
+      "step": 1000
+    },
+    {
+      "epoch": 4.63316002310803,
+      "grad_norm": 0.43812036514282227,
+      "learning_rate": 4.123421664314483e-05,
+      "loss": 0.1806,
+      "step": 1005
+    },
+    {
+      "epoch": 4.6562680531484695,
+      "grad_norm": 0.48830482363700867,
+      "learning_rate": 4.076571050006036e-05,
+      "loss": 0.1827,
+      "step": 1010
+    },
+    {
+      "epoch": 4.679376083188908,
+      "grad_norm": 0.5706621408462524,
+      "learning_rate": 4.0298042269428225e-05,
+      "loss": 0.1846,
+      "step": 1015
+    },
+    {
+      "epoch": 4.702484113229347,
+      "grad_norm": 0.4595714807510376,
+      "learning_rate": 3.983125438711032e-05,
+      "loss": 0.178,
+      "step": 1020
+    },
+    {
+      "epoch": 4.725592143269786,
+      "grad_norm": 0.6200171113014221,
+      "learning_rate": 3.9365389209086415e-05,
+      "loss": 0.1755,
+      "step": 1025
+    },
+    {
+      "epoch": 4.748700173310225,
+      "grad_norm": 0.4130564033985138,
+      "learning_rate": 3.890048900761076e-05,
+      "loss": 0.1759,
+      "step": 1030
+    },
+    {
+      "epoch": 4.771808203350664,
+      "grad_norm": 0.8717600107192993,
+      "learning_rate": 3.8436595967376404e-05,
+      "loss": 0.1802,
+      "step": 1035
+    },
+    {
+      "epoch": 4.7949162333911035,
+      "grad_norm": 0.5592686533927917,
+      "learning_rate": 3.7973752181687335e-05,
+      "loss": 0.1897,
+      "step": 1040
+    },
+    {
+      "epoch": 4.818024263431543,
+      "grad_norm": 0.6098639965057373,
+      "learning_rate": 3.751199964863893e-05,
+      "loss": 0.1824,
+      "step": 1045
+    },
+    {
+      "epoch": 4.841132293471982,
+      "grad_norm": 0.6093428134918213,
+      "learning_rate": 3.7051380267307214e-05,
+      "loss": 0.1835,
+      "step": 1050
+    },
+    {
+      "epoch": 4.864240323512421,
+      "grad_norm": 0.45407184958457947,
+      "learning_rate": 3.659193583394681e-05,
+      "loss": 0.178,
+      "step": 1055
+    },
+    {
+      "epoch": 4.887348353552859,
+      "grad_norm": 0.5157074332237244,
+      "learning_rate": 3.613370803819846e-05,
+      "loss": 0.1795,
+      "step": 1060
+    },
+    {
+      "epoch": 4.910456383593298,
+      "grad_norm": 0.529229998588562,
+      "learning_rate": 3.567673845930616e-05,
+      "loss": 0.1845,
+      "step": 1065
+    },
+    {
+      "epoch": 4.9335644136337375,
+      "grad_norm": 0.5734789967536926,
+      "learning_rate": 3.522106856234423e-05,
+      "loss": 0.1784,
+      "step": 1070
+    },
+    {
+      "epoch": 4.956672443674177,
+      "grad_norm": 0.5396260023117065,
+      "learning_rate": 3.476673969445484e-05,
+      "loss": 0.1804,
+      "step": 1075
+    },
+    {
+      "epoch": 4.979780473714616,
+      "grad_norm": 0.42490893602371216,
+      "learning_rate": 3.43137930810962e-05,
+      "loss": 0.1863,
+      "step": 1080
+    },
+    {
+      "epoch": 5.0,
+      "grad_norm": 0.5912376642227173,
+      "learning_rate": 3.3862269822301765e-05,
+      "loss": 0.1736,
+      "step": 1085
+    },
+    {
+      "epoch": 5.023108030040439,
+      "grad_norm": 0.4906028211116791,
+      "learning_rate": 3.3412210888950926e-05,
+      "loss": 0.1791,
+      "step": 1090
+    },
+    {
+      "epoch": 5.046216060080878,
+      "grad_norm": 0.49465078115463257,
+      "learning_rate": 3.2963657119051226e-05,
+      "loss": 0.1736,
+      "step": 1095
+    },
+    {
+      "epoch": 5.0693240901213175,
+      "grad_norm": 0.508440375328064,
+      "learning_rate": 3.251664921403288e-05,
+      "loss": 0.1728,
+      "step": 1100
+    },
+    {
+      "epoch": 5.092432120161757,
+      "grad_norm": 0.5910377502441406,
+      "learning_rate": 3.2071227735055435e-05,
+      "loss": 0.1713,
+      "step": 1105
+    },
+    {
+      "epoch": 5.115540150202195,
+      "grad_norm": 0.4319545030593872,
+      "learning_rate": 3.162743309932735e-05,
+      "loss": 0.1622,
+      "step": 1110
+    },
+    {
+      "epoch": 5.138648180242634,
+      "grad_norm": 0.4956379234790802,
+      "learning_rate": 3.118530557643859e-05,
+      "loss": 0.1698,
+      "step": 1115
+    },
+    {
+      "epoch": 5.161756210283073,
+      "grad_norm": 0.6078504323959351,
+      "learning_rate": 3.074488528470651e-05,
+      "loss": 0.172,
+      "step": 1120
+    },
+    {
+      "epoch": 5.184864240323512,
+      "grad_norm": 0.46639910340309143,
+      "learning_rate": 3.0306212187535653e-05,
+      "loss": 0.166,
+      "step": 1125
+    },
+    {
+      "epoch": 5.2079722703639515,
+      "grad_norm": 0.5496683716773987,
+      "learning_rate": 2.9869326089791384e-05,
+      "loss": 0.169,
+      "step": 1130
+    },
+    {
+      "epoch": 5.231080300404391,
+      "grad_norm": 0.4343482255935669,
+      "learning_rate": 2.9434266634188158e-05,
+      "loss": 0.1745,
+      "step": 1135
+    },
+    {
+      "epoch": 5.25418833044483,
+      "grad_norm": 0.6727978587150574,
+      "learning_rate": 2.900107329769224e-05,
+      "loss": 0.1617,
+      "step": 1140
+    },
+    {
+      "epoch": 5.277296360485269,
+      "grad_norm": 0.47565704584121704,
+      "learning_rate": 2.856978538793972e-05,
+      "loss": 0.1683,
+      "step": 1145
+    },
+    {
+      "epoch": 5.300404390525708,
+      "grad_norm": 0.5185399055480957,
+      "learning_rate": 2.8140442039669647e-05,
+      "loss": 0.1742,
+      "step": 1150
+    },
+    {
+      "epoch": 5.323512420566146,
+      "grad_norm": 0.6550673246383667,
+      "learning_rate": 2.771308221117309e-05,
+      "loss": 0.1641,
+      "step": 1155
+    },
+    {
+      "epoch": 5.3466204506065855,
+      "grad_norm": 0.34338533878326416,
+      "learning_rate": 2.7287744680757954e-05,
+      "loss": 0.1758,
+      "step": 1160
+    },
+    {
+      "epoch": 5.369728480647025,
+      "grad_norm": 0.4614684581756592,
+      "learning_rate": 2.686446804323044e-05,
+      "loss": 0.1709,
+      "step": 1165
+    },
+    {
+      "epoch": 5.392836510687464,
+      "grad_norm": 0.4567757248878479,
+      "learning_rate": 2.6443290706392776e-05,
+      "loss": 0.1701,
+      "step": 1170
+    },
+    {
+      "epoch": 5.415944540727903,
+      "grad_norm": 0.49725341796875,
+      "learning_rate": 2.602425088755831e-05,
+      "loss": 0.1776,
+      "step": 1175
+    },
+    {
+      "epoch": 5.439052570768342,
+      "grad_norm": 0.5264298915863037,
+      "learning_rate": 2.560738661008364e-05,
+      "loss": 0.1674,
+      "step": 1180
+    },
+    {
+      "epoch": 5.462160600808781,
+      "grad_norm": 0.3638445734977722,
+      "learning_rate": 2.5192735699918286e-05,
+      "loss": 0.169,
+      "step": 1185
+    },
+    {
+      "epoch": 5.4852686308492205,
+      "grad_norm": 0.4739927649497986,
+      "learning_rate": 2.478033578217258e-05,
+      "loss": 0.1763,
+      "step": 1190
+    },
+    {
+      "epoch": 5.50837666088966,
+      "grad_norm": 0.5547608733177185,
+      "learning_rate": 2.4370224277703484e-05,
+      "loss": 0.1769,
+      "step": 1195
+    },
+    {
+      "epoch": 5.531484690930098,
+      "grad_norm": 0.49245351552963257,
+      "learning_rate": 2.3962438399719005e-05,
+      "loss": 0.174,
+      "step": 1200
+    },
+    {
+      "epoch": 5.554592720970537,
+      "grad_norm": 0.607456386089325,
+      "learning_rate": 2.355701515040169e-05,
+      "loss": 0.1718,
+      "step": 1205
+    },
+    {
+      "epoch": 5.577700751010976,
+      "grad_norm": 0.6700326204299927,
+      "learning_rate": 2.315399131755081e-05,
+      "loss": 0.1687,
+      "step": 1210
+    },
+    {
+      "epoch": 5.600808781051415,
+      "grad_norm": 0.5112605094909668,
+      "learning_rate": 2.2753403471244483e-05,
+      "loss": 0.1727,
+      "step": 1215
+    },
+    {
+      "epoch": 5.6239168110918545,
+      "grad_norm": 0.5395715236663818,
+      "learning_rate": 2.2355287960521197e-05,
+      "loss": 0.1801,
+      "step": 1220
+    },
+    {
+      "epoch": 5.647024841132294,
+      "grad_norm": 0.48670148849487305,
+      "learning_rate": 2.195968091008162e-05,
+      "loss": 0.1634,
+      "step": 1225
+    },
+    {
+      "epoch": 5.670132871172733,
+      "grad_norm": 0.6344140768051147,
+      "learning_rate": 2.156661821701062e-05,
+      "loss": 0.1688,
+      "step": 1230
+    },
+    {
+      "epoch": 5.693240901213172,
+      "grad_norm": 1.38614022731781,
+      "learning_rate": 2.117613554751996e-05,
+      "loss": 0.1674,
+      "step": 1235
+    },
+    {
+      "epoch": 5.716348931253611,
+      "grad_norm": 0.40783703327178955,
+      "learning_rate": 2.0788268333712063e-05,
+      "loss": 0.1702,
+      "step": 1240
+    },
+    {
+      "epoch": 5.739456961294049,
+      "grad_norm": 0.506542444229126,
+      "learning_rate": 2.0403051770364872e-05,
+      "loss": 0.1678,
+      "step": 1245
+    },
+    {
+      "epoch": 5.7625649913344885,
+      "grad_norm": 0.47708380222320557,
+      "learning_rate": 2.0020520811738254e-05,
+      "loss": 0.1745,
+      "step": 1250
+    },
+    {
+      "epoch": 5.785673021374928,
+      "grad_norm": 0.6139983534812927,
+      "learning_rate": 1.9640710168402432e-05,
+      "loss": 0.1686,
+      "step": 1255
+    },
+    {
+      "epoch": 5.808781051415367,
+      "grad_norm": 0.42571625113487244,
+      "learning_rate": 1.926365430408818e-05,
+      "loss": 0.1681,
+      "step": 1260
+    },
+    {
+      "epoch": 5.831889081455806,
+      "grad_norm": 0.4261309504508972,
+      "learning_rate": 1.8889387432559774e-05,
+      "loss": 0.1754,
+      "step": 1265
+    },
+    {
+      "epoch": 5.854997111496245,
+      "grad_norm": 0.5212744474411011,
+      "learning_rate": 1.851794351451033e-05,
+      "loss": 0.1771,
+      "step": 1270
+    },
+    {
+      "epoch": 5.878105141536684,
+      "grad_norm": 0.6218976974487305,
+      "learning_rate": 1.814935625448034e-05,
+      "loss": 0.1795,
+      "step": 1275
+    },
+    {
+      "epoch": 5.9012131715771226,
+      "grad_norm": 0.5627200603485107,
+      "learning_rate": 1.778365909779932e-05,
+      "loss": 0.1749,
+      "step": 1280
+    },
+    {
+      "epoch": 5.924321201617563,
+      "grad_norm": 0.39734962582588196,
+      "learning_rate": 1.7420885227550936e-05,
+      "loss": 0.1754,
+      "step": 1285
+    },
+    {
+      "epoch": 5.947429231658001,
+      "grad_norm": 0.5245198011398315,
+      "learning_rate": 1.706106756156209e-05,
+      "loss": 0.1761,
+      "step": 1290
+    },
+    {
+      "epoch": 5.97053726169844,
+      "grad_norm": 0.8322150707244873,
+      "learning_rate": 1.6704238749415957e-05,
+      "loss": 0.1692,
+      "step": 1295
+    },
+    {
+      "epoch": 5.993645291738879,
+      "grad_norm": 0.5423708558082581,
+      "learning_rate": 1.6350431169489304e-05,
+      "loss": 0.1695,
+      "step": 1300
+    },
+    {
+      "epoch": 6.013864818024263,
+      "grad_norm": 0.47064247727394104,
+      "learning_rate": 1.5999676926014683e-05,
+      "loss": 0.1659,
+      "step": 1305
+    },
+    {
+      "epoch": 6.0369728480647025,
+      "grad_norm": 0.4340321719646454,
+      "learning_rate": 1.5652007846167088e-05,
+      "loss": 0.1567,
+      "step": 1310
+    },
+    {
+      "epoch": 6.060080878105142,
+      "grad_norm": 0.4804087281227112,
+      "learning_rate": 1.530745547717618e-05,
+      "loss": 0.1637,
+      "step": 1315
+    },
+    {
+      "epoch": 6.083188908145581,
+      "grad_norm": 0.442632794380188,
+      "learning_rate": 1.4966051083463555e-05,
+      "loss": 0.1735,
+      "step": 1320
+    },
+    {
+      "epoch": 6.10629693818602,
+      "grad_norm": 0.4208667278289795,
+      "learning_rate": 1.4627825643805981e-05,
+      "loss": 0.1685,
+      "step": 1325
+    },
+    {
+      "epoch": 6.129404968226459,
+      "grad_norm": 0.7900300025939941,
+      "learning_rate": 1.429280984852432e-05,
+      "loss": 0.1716,
+      "step": 1330
+    },
+    {
+      "epoch": 6.152512998266897,
+      "grad_norm": 0.4661615490913391,
+      "learning_rate": 1.3961034096698683e-05,
+      "loss": 0.1664,
+      "step": 1335
+    },
+    {
+      "epoch": 6.1756210283073365,
+      "grad_norm": 0.42455703020095825,
+      "learning_rate": 1.3632528493410141e-05,
+      "loss": 0.1585,
+      "step": 1340
+    },
+    {
+      "epoch": 6.198729058347776,
+      "grad_norm": 0.4959942698478699,
+      "learning_rate": 1.3307322847008946e-05,
+      "loss": 0.1644,
+      "step": 1345
+    },
+    {
+      "epoch": 6.221837088388215,
+      "grad_norm": 0.5466529726982117,
+      "learning_rate": 1.2985446666409717e-05,
+      "loss": 0.1685,
+      "step": 1350
+    },
+    {
+      "epoch": 6.244945118428654,
+      "grad_norm": 0.47902652621269226,
+      "learning_rate": 1.266692915841391e-05,
+      "loss": 0.1719,
+      "step": 1355
+    },
+    {
+      "epoch": 6.268053148469093,
+      "grad_norm": 0.46042776107788086,
+      "learning_rate": 1.235179922505954e-05,
+      "loss": 0.1656,
+      "step": 1360
+    },
+    {
+      "epoch": 6.291161178509532,
+      "grad_norm": 0.5602250695228577,
+      "learning_rate": 1.2040085460998673e-05,
+      "loss": 0.1654,
+      "step": 1365
+    },
+    {
+      "epoch": 6.314269208549971,
+      "grad_norm": 0.4146384298801422,
+      "learning_rate": 1.1731816150902752e-05,
+      "loss": 0.1604,
+      "step": 1370
+    },
+    {
+      "epoch": 6.337377238590411,
+      "grad_norm": 0.5237134695053101,
+      "learning_rate": 1.1427019266896044e-05,
+      "loss": 0.1642,
+      "step": 1375
+    },
+    {
+      "epoch": 6.360485268630849,
+      "grad_norm": 0.5266466736793518,
+      "learning_rate": 1.1125722466017547e-05,
+      "loss": 0.1636,
+      "step": 1380
+    },
+    {
+      "epoch": 6.383593298671288,
+      "grad_norm": 0.8678638339042664,
+      "learning_rate": 1.082795308771129e-05,
+      "loss": 0.1636,
+      "step": 1385
+    },
+    {
+      "epoch": 6.406701328711727,
+      "grad_norm": 0.4591081440448761,
+      "learning_rate": 1.0533738151345707e-05,
+      "loss": 0.1642,
+      "step": 1390
+    },
+    {
+      "epoch": 6.429809358752166,
+      "grad_norm": 0.4099045991897583,
+      "learning_rate": 1.0243104353761846e-05,
+      "loss": 0.1733,
+      "step": 1395
+    },
+    {
+      "epoch": 6.4529173887926055,
+      "grad_norm": 0.4741394519805908,
+      "learning_rate": 9.956078066850877e-06,
+      "loss": 0.1643,
+      "step": 1400
+    },
+    {
+      "epoch": 6.476025418833045,
+      "grad_norm": 0.4387037754058838,
+      "learning_rate": 9.672685335161236e-06,
+      "loss": 0.161,
+      "step": 1405
+    },
+    {
+      "epoch": 6.499133448873484,
+      "grad_norm": 0.4587726294994354,
+      "learning_rate": 9.392951873535282e-06,
+      "loss": 0.1617,
+      "step": 1410
+    },
+    {
+      "epoch": 6.522241478913923,
+      "grad_norm": 0.49484848976135254,
+      "learning_rate": 9.116903064775966e-06,
+      "loss": 0.1656,
+      "step": 1415
+    },
+    {
+      "epoch": 6.545349508954361,
+      "grad_norm": 0.48660051822662354,
+      "learning_rate": 8.844563957343644e-06,
+      "loss": 0.1655,
+      "step": 1420
+    },
+    {
+      "epoch": 6.5684575389948,
+      "grad_norm": 0.4600284695625305,
+      "learning_rate": 8.57595926308314e-06,
+      "loss": 0.1661,
+      "step": 1425
+    },
+    {
+      "epoch": 6.5915655690352395,
+      "grad_norm": 0.5686083436012268,
+      "learning_rate": 8.31111335498148e-06,
+      "loss": 0.1701,
+      "step": 1430
+    },
+    {
+      "epoch": 6.614673599075679,
+      "grad_norm": 0.4815639853477478,
+      "learning_rate": 8.050050264956256e-06,
+      "loss": 0.1631,
+      "step": 1435
+    },
+    {
+      "epoch": 6.637781629116118,
+      "grad_norm": 0.5048244595527649,
+      "learning_rate": 7.792793681675015e-06,
+      "loss": 0.1589,
+      "step": 1440
+    },
+    {
+      "epoch": 6.660889659156557,
+      "grad_norm": 0.38909146189689636,
+      "learning_rate": 7.539366948405768e-06,
+      "loss": 0.1635,
+      "step": 1445
+    },
+    {
+      "epoch": 6.683997689196996,
+      "grad_norm": 0.4918948709964752,
+      "learning_rate": 7.289793060898792e-06,
+      "loss": 0.1657,
+      "step": 1450
+    },
+    {
+      "epoch": 6.707105719237435,
+      "grad_norm": 1.3107045888900757,
+      "learning_rate": 7.044094665300088e-06,
+      "loss": 0.1711,
+      "step": 1455
+    },
+    {
+      "epoch": 6.730213749277874,
+      "grad_norm": 0.5031064748764038,
+      "learning_rate": 6.802294056096448e-06,
+      "loss": 0.1706,
+      "step": 1460
+    },
+    {
+      "epoch": 6.753321779318313,
+      "grad_norm": 0.4544471800327301,
+      "learning_rate": 6.564413174092443e-06,
+      "loss": 0.1583,
+      "step": 1465
+    },
+    {
+      "epoch": 6.776429809358752,
+      "grad_norm": 0.4446888267993927,
+      "learning_rate": 6.330473604419601e-06,
+      "loss": 0.1604,
+      "step": 1470
+    },
+    {
+      "epoch": 6.799537839399191,
+      "grad_norm": 0.4598006308078766,
+      "learning_rate": 6.100496574577691e-06,
+      "loss": 0.1665,
+      "step": 1475
+    },
+    {
+      "epoch": 6.82264586943963,
+      "grad_norm": 0.4443235695362091,
+      "learning_rate": 5.874502952508676e-06,
+      "loss": 0.1648,
+      "step": 1480
+    },
+    {
+      "epoch": 6.845753899480069,
+      "grad_norm": 0.5368012189865112,
+      "learning_rate": 5.65251324470304e-06,
+      "loss": 0.1704,
+      "step": 1485
+    },
+    {
+      "epoch": 6.8688619295205084,
+      "grad_norm": 0.45232707262039185,
+      "learning_rate": 5.434547594339173e-06,
+      "loss": 0.1652,
+      "step": 1490
+    },
+    {
+      "epoch": 6.891969959560948,
+      "grad_norm": 0.4863213002681732,
+      "learning_rate": 5.220625779455501e-06,
+      "loss": 0.1651,
+      "step": 1495
+    },
+    {
+      "epoch": 6.915077989601387,
+      "grad_norm": 0.45794785022735596,
+      "learning_rate": 5.01076721115587e-06,
+      "loss": 0.1601,
+      "step": 1500
+    },
+    {
+      "epoch": 6.938186019641826,
+      "grad_norm": 0.46603628993034363,
+      "learning_rate": 4.804990931848219e-06,
+      "loss": 0.165,
+      "step": 1505
+    },
+    {
+      "epoch": 6.961294049682264,
+      "grad_norm": 0.48978325724601746,
+      "learning_rate": 4.6033156135166565e-06,
+      "loss": 0.1645,
+      "step": 1510
+    },
+    {
+      "epoch": 6.984402079722703,
+      "grad_norm": 0.5197871923446655,
+      "learning_rate": 4.4057595560271945e-06,
+      "loss": 0.1627,
+      "step": 1515
+    },
+    {
+      "epoch": 7.0046216060080875,
+      "grad_norm": 0.5848413109779358,
+      "learning_rate": 4.212340685467242e-06,
+      "loss": 0.1679,
+      "step": 1520
+    },
+    {
+      "epoch": 7.027729636048527,
+      "grad_norm": 0.3766275644302368,
+      "learning_rate": 4.023076552518951e-06,
+      "loss": 0.1641,
+      "step": 1525
+    },
+    {
+      "epoch": 7.050837666088966,
+      "grad_norm": 0.4636153280735016,
+      "learning_rate": 3.837984330866767e-06,
+      "loss": 0.1655,
+      "step": 1530
+    },
+    {
+      "epoch": 7.073945696129405,
+      "grad_norm": 0.4446038007736206,
+      "learning_rate": 3.6570808156390246e-06,
+      "loss": 0.1711,
+      "step": 1535
+    },
+    {
+      "epoch": 7.097053726169844,
+      "grad_norm": 0.7249389886856079,
+      "learning_rate": 3.4803824218840265e-06,
+      "loss": 0.1658,
+      "step": 1540
+    },
+    {
+      "epoch": 7.120161756210283,
+      "grad_norm": 0.6228806376457214,
+      "learning_rate": 3.3079051830805172e-06,
+      "loss": 0.1622,
+      "step": 1545
+    },
+    {
+      "epoch": 7.143269786250722,
+      "grad_norm": 0.3605245053768158,
+      "learning_rate": 3.1396647496828247e-06,
+      "loss": 0.1634,
+      "step": 1550
+    },
+    {
+      "epoch": 7.1663778162911616,
+      "grad_norm": 0.6837649345397949,
+      "learning_rate": 2.97567638770076e-06,
+      "loss": 0.165,
+      "step": 1555
+    },
+    {
+      "epoch": 7.1894858463316,
+      "grad_norm": 0.8120383024215698,
+      "learning_rate": 2.8159549773144058e-06,
+      "loss": 0.1678,
+      "step": 1560
+    },
+    {
+      "epoch": 7.212593876372039,
+      "grad_norm": 0.4651970863342285,
+      "learning_rate": 2.660515011523845e-06,
+      "loss": 0.1643,
+      "step": 1565
+    },
+    {
+      "epoch": 7.235701906412478,
+      "grad_norm": 0.5014097094535828,
+      "learning_rate": 2.509370594834154e-06,
+      "loss": 0.1622,
+      "step": 1570
+    },
+    {
+      "epoch": 7.258809936452917,
+      "grad_norm": 0.5164337754249573,
+      "learning_rate": 2.3625354419754884e-06,
+      "loss": 0.1619,
+      "step": 1575
+    },
+    {
+      "epoch": 7.281917966493356,
+      "grad_norm": 0.538179874420166,
+      "learning_rate": 2.2200228766586896e-06,
+      "loss": 0.1572,
+      "step": 1580
+    },
+    {
+      "epoch": 7.305025996533796,
+      "grad_norm": 0.4584835171699524,
+      "learning_rate": 2.0818458303662525e-06,
+      "loss": 0.1658,
+      "step": 1585
+    },
+    {
+      "epoch": 7.328134026574235,
+      "grad_norm": 0.47576990723609924,
+      "learning_rate": 1.9480168411789567e-06,
+      "loss": 0.1553,
+      "step": 1590
+    },
+    {
+      "epoch": 7.351242056614674,
+      "grad_norm": 0.47631341218948364,
+      "learning_rate": 1.8185480526381803e-06,
+      "loss": 0.1584,
+      "step": 1595
+    },
+    {
+      "epoch": 7.374350086655113,
+      "grad_norm": 0.46469229459762573,
+      "learning_rate": 1.693451212643965e-06,
+      "loss": 0.158,
+      "step": 1600
+    },
+    {
+      "epoch": 7.397458116695551,
+      "grad_norm": 0.6343350410461426,
+      "learning_rate": 1.5727376723890575e-06,
+      "loss": 0.1638,
+      "step": 1605
+    },
+    {
+      "epoch": 7.4205661467359905,
+      "grad_norm": 0.520357608795166,
+      "learning_rate": 1.4564183853288949e-06,
+      "loss": 0.1648,
+      "step": 1610
+    },
+    {
+      "epoch": 7.44367417677643,
+      "grad_norm": 0.5116673111915588,
+      "learning_rate": 1.3445039061877107e-06,
+      "loss": 0.1664,
+      "step": 1615
+    },
+    {
+      "epoch": 7.466782206816869,
+      "grad_norm": 0.5481125116348267,
+      "learning_rate": 1.2370043900007789e-06,
+      "loss": 0.162,
+      "step": 1620
+    },
+    {
+      "epoch": 7.489890236857308,
+      "grad_norm": 0.3400834798812866,
+      "learning_rate": 1.1339295911929782e-06,
+      "loss": 0.1619,
+      "step": 1625
+    },
+    {
+      "epoch": 7.512998266897747,
+      "grad_norm": 0.4965187609195709,
+      "learning_rate": 1.0352888626936786e-06,
+      "loss": 0.1575,
+      "step": 1630
+    },
+    {
+      "epoch": 7.536106296938186,
+      "grad_norm": 1.096940040588379,
+      "learning_rate": 9.410911550880475e-07,
+      "loss": 0.1599,
+      "step": 1635
+    },
+    {
+      "epoch": 7.559214326978625,
+      "grad_norm": 0.5760189890861511,
+      "learning_rate": 8.513450158049108e-07,
+      "loss": 0.1528,
+      "step": 1640
+    },
+    {
+      "epoch": 7.5823223570190645,
+      "grad_norm": 0.6139164566993713,
+      "learning_rate": 7.660585883411397e-07,
+      "loss": 0.1618,
+      "step": 1645
+    },
+    {
+      "epoch": 7.605430387059503,
+      "grad_norm": 0.42764243483543396,
+      "learning_rate": 6.852396115227199e-07,
+      "loss": 0.1596,
+      "step": 1650
+    },
+    {
+      "epoch": 7.628538417099942,
+      "grad_norm": 0.4714425504207611,
+      "learning_rate": 6.08895418802552e-07,
+      "loss": 0.1632,
+      "step": 1655
+    },
+    {
+      "epoch": 7.651646447140381,
+      "grad_norm": 0.46559858322143555,
+      "learning_rate": 5.370329375949945e-07,
+      "loss": 0.1661,
+      "step": 1660
+    },
+    {
+      "epoch": 7.67475447718082,
+      "grad_norm": 0.49914413690567017,
+      "learning_rate": 4.6965868864730643e-07,
+      "loss": 0.1651,
+      "step": 1665
+    },
+    {
+      "epoch": 7.697862507221259,
+      "grad_norm": 0.6735681295394897,
+      "learning_rate": 4.06778785447931e-07,
+      "loss": 0.1633,
+      "step": 1670
+    },
+    {
+      "epoch": 7.720970537261699,
+      "grad_norm": 0.4275800287723541,
+      "learning_rate": 3.4839893367176745e-07,
+      "loss": 0.158,
+      "step": 1675
+    },
+    {
+      "epoch": 7.744078567302138,
+      "grad_norm": 0.48241615295410156,
+      "learning_rate": 2.945244306624628e-07,
+      "loss": 0.158,
+      "step": 1680
+    },
+    {
+      "epoch": 7.767186597342577,
+      "grad_norm": 0.4567784070968628,
+      "learning_rate": 2.451601649517188e-07,
+      "loss": 0.1636,
+      "step": 1685
+    },
+    {
+      "epoch": 7.790294627383016,
+      "grad_norm": 0.45031699538230896,
+      "learning_rate": 2.0031061581570753e-07,
+      "loss": 0.1617,
+      "step": 1690
+    },
+    {
+      "epoch": 7.813402657423454,
+      "grad_norm": 0.4920361340045929,
+      "learning_rate": 1.5997985286863028e-07,
+      "loss": 0.161,
+      "step": 1695
+    },
+    {
+      "epoch": 7.8365106874638935,
+      "grad_norm": 0.43548688292503357,
+      "learning_rate": 1.2417153569344032e-07,
+      "loss": 0.1587,
+      "step": 1700
+    },
+    {
+      "epoch": 7.859618717504333,
+      "grad_norm": 0.4531209170818329,
+      "learning_rate": 9.288891350979767e-08,
+      "loss": 0.1634,
+      "step": 1705
+    },
+    {
+      "epoch": 7.882726747544772,
+      "grad_norm": 0.43480750918388367,
+      "learning_rate": 6.613482487918821e-08,
+      "loss": 0.1625,
+      "step": 1710
+    },
+    {
+      "epoch": 7.905834777585211,
+      "grad_norm": 0.5283259153366089,
+      "learning_rate": 4.391169744741297e-08,
+      "loss": 0.1626,
+      "step": 1715
+    },
+    {
+      "epoch": 7.92894280762565,
+      "grad_norm": 0.5982206463813782,
+      "learning_rate": 2.6221547724253337e-08,
+      "loss": 0.1671,
+      "step": 1720
+    },
+    {
+      "epoch": 7.952050837666089,
+      "grad_norm": 0.3607623875141144,
+      "learning_rate": 1.3065980900539565e-08,
+      "loss": 0.1598,
+      "step": 1725
+    },
+    {
+      "epoch": 7.975158867706528,
+      "grad_norm": 0.4897676110267639,
+      "learning_rate": 4.446190702461772e-09,
+      "loss": 0.1614,
+      "step": 1730
+    },
+    {
+      "epoch": 7.9982668977469675,
+      "grad_norm": 0.5443258285522461,
+      "learning_rate": 3.6295928326768313e-10,
+      "loss": 0.1597,
+      "step": 1735
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 1736,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2.8225736231459226e+19,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Qwen3-4B-Instruct-2507-sft-fusang-swa=2k_sink=0_fadec/checkpoint-1760/tokenizer_config.json

@@ -0,0 +1,239 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151665": {
+      "content": "<tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151666": {
+      "content": "</tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151667": {
+      "content": "<think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151668": {
+      "content": "</think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 1010000,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

Qwen3-4B-Instruct-2507-sft-fusang-swa=2k_sink=0_falayer=13_from1_step4_fadec/checkpoint-921/README.md

@@ -0,0 +1,207 @@
+---
+base_model: /share/models/Qwen3-4B-Instruct-2507
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/share/models/Qwen3-4B-Instruct-2507
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1

Qwen3-4B-Instruct-2507-sft-fusang-swa=2k_sink=0_falayer=13_from1_step4_fadec/checkpoint-921/adapter_config.json

@@ -0,0 +1,38 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/share/models/Qwen3-4B-Instruct-2507",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "v_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": true
+}

Qwen3-4B-Instruct-2507-sft-fusang-swa=2k_sink=0_falayer=13_from1_step4_fadec/checkpoint-921/added_tokens.json

@@ -0,0 +1,28 @@
+{
+  "</think>": 151668,
+  "</tool_call>": 151658,
+  "</tool_response>": 151666,
+  "<think>": 151667,
+  "<tool_call>": 151657,
+  "<tool_response>": 151665,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

Qwen3-4B-Instruct-2507-sft-fusang-swa=2k_sink=0_falayer=13_from1_step4_fadec/checkpoint-921/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Qwen3-4B-Instruct-2507-sft-fusang-swa=2k_sink=0_falayer=13_from1_step4_fadec/checkpoint-921/tokenizer_config.json

@@ -0,0 +1,239 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151665": {
+      "content": "<tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151666": {
+      "content": "</tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151667": {
+      "content": "<think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151668": {
+      "content": "</think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 1010000,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

Qwen3-4B-Instruct-2507-sft-fusang-swa=2k_sink=0_falayer=25_from1_step2/checkpoint-921/chat_template.jinja

@@ -0,0 +1,61 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if message.content is string %}
+        {%- set content = message.content %}
+    {%- else %}
+        {%- set content = '' %}
+    {%- endif %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

Qwen3-4B-Instruct-2507-sft-fusang-swa=2k_sink=0_falayer=25_from1_step2/checkpoint-921/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Qwen3-4B-Instruct-2507-sft-fusang-swa=2k_sink=0_falayer=25_from1_step2/checkpoint-921/tokenizer_config.json

@@ -0,0 +1,239 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151665": {
+      "content": "<tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151666": {
+      "content": "</tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151667": {
+      "content": "<think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151668": {
+      "content": "</think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 1010000,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

Qwen3-4B-Instruct-2507-sft-fusang-swa=2k_sink=0_falayer=7_from0_step8_fadec/checkpoint-1085/adapter_config.json

@@ -0,0 +1,38 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/share/models/Qwen3-4B-Instruct-2507",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "v_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": true
+}

Qwen3-4B-Instruct-2507-sft-fusang-swa=2k_sink=0_falayer=7_from0_step8_fadec/checkpoint-1085/added_tokens.json

@@ -0,0 +1,28 @@
+{
+  "</think>": 151668,
+  "</tool_call>": 151658,
+  "</tool_response>": 151666,
+  "<think>": 151667,
+  "<tool_call>": 151657,
+  "<tool_response>": 151665,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

Qwen3-4B-Instruct-2507-sft-fusang-swa=2k_sink=0_falayer=7_from0_step8_fadec/checkpoint-1085/chat_template.jinja

@@ -0,0 +1,61 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if message.content is string %}
+        {%- set content = message.content %}
+    {%- else %}
+        {%- set content = '' %}
+    {%- endif %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

Qwen3-4B-Instruct-2507-sft-fusang-swa=2k_sink=0_falayer=7_from0_step8_fadec/checkpoint-1085/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Qwen3-4B-Instruct-2507-sft-fusang-swa=2k_sink=0_falayer=7_from0_step8_fadec/checkpoint-1085/tokenizer_config.json

@@ -0,0 +1,239 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151665": {
+      "content": "<tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151666": {
+      "content": "</tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151667": {
+      "content": "<think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151668": {
+      "content": "</think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 1010000,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

Qwen3-4B-Instruct-2507-sft-fusang-swa=2k_sink=10/checkpoint-1760/adapter_config.json

@@ -0,0 +1,38 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/share/models/Qwen3-4B-Instruct-2507",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "q_proj",
+    "v_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": true
+}

Qwen3-4B-Instruct-2507-sft-fusang-swa=2k_sink=10/checkpoint-1760/added_tokens.json

@@ -0,0 +1,28 @@
+{
+  "</think>": 151668,
+  "</tool_call>": 151658,
+  "</tool_response>": 151666,
+  "<think>": 151667,
+  "<tool_call>": 151657,
+  "<tool_response>": 151665,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

Qwen3-4B-Instruct-2507-sft-fusang-swa=2k_sink=100/checkpoint-1760/adapter_config.json

@@ -0,0 +1,38 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/share/models/Qwen3-4B-Instruct-2507",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "k_proj",
+    "v_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": true
+}

Qwen3-4B-Instruct-2507-sft-fusang-swa=2k_sink=100/checkpoint-1760/added_tokens.json

@@ -0,0 +1,28 @@
+{
+  "</think>": 151668,
+  "</tool_call>": 151658,
+  "</tool_response>": 151666,
+  "<think>": 151667,
+  "<tool_call>": 151657,
+  "<tool_response>": 151665,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

Qwen3-4B-Instruct-2507-sft-fusang-swa=2k_sink=100/checkpoint-1760/chat_template.jinja

@@ -0,0 +1,61 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if message.content is string %}
+        {%- set content = message.content %}
+    {%- else %}
+        {%- set content = '' %}
+    {%- endif %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}