Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+assets/Plano-Orchestrator.png filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

LICENSE

@@ -0,0 +1,89 @@
+# Katanemo Labs, Inc. COMMUNITY LICENSE AGREEMENT  
+**Version Release Date:** April 11th, 2025  
+
+This Katanemo Labs, Inc. COMMUNITY LICENSE AGREEMENT is based on the Llama 3.2 Community License, Copyright © Meta Platforms, Inc. The terms and conditions have been adapted to reflect the proprietary nature of Katanemo Labs' materials.
+
+---
+
+1.Definitions
+  a. "Agreement": The terms and conditions for use, reproduction, distribution, and modification of the Katanemo Materials set forth herein.
+  b. "Documentation": The specifications, manuals, and documentation accompanying Katanemo LLMs v1.
+  c. "Licensee" or "you: The individual or entity entering into this Agreement, including your employer if you are acting on their behalf.
+  d. "Katanemo": The foundational large language models and software provided by Katanemo Labs, Inc., available at https://huggingface.co/katanemolabs.
+  e. "Katanemo Materials": Collectively, Katanemo's proprietary models and Documentation. Some Materials are derived from the Qwen language models licensed under the Qwen RESEARCH LICENSE AGREEMENT.
+  f. "Katanemo Labs" or "we": Katanemo Labs Inc., a Delaware, USA Corporation.
+
+---
+
+2. 
+By clicking "I Accept" or using any part of the Katanemo Materials, you agree to be bound by this Agreement.
+
+---
+
+3. LICENSE RIGHTS AND REDISTRIBUTION
+  a. Grant of Rights  
+  Subject to the restrictions in Section 4, you are granted a non-exclusive, worldwide, non-transferable, and royalty-free license to:
+  - Use, reproduce, distribute, and modify the Katanemo Materials.
+  - Create derivative works based on the Katanemo Materials.
+  
+  b. Permitted Redistribution  
+  If you distribute the Katanemo Materials or any derivative work:
+  - You must include a copy of this License.
+  - You must prominently display the notice “Built with Katanemo” on a related website or documentation.
+  
+  c. Attribution Requirement  
+  You must include the following attribution notice in any distributed or public-facing use:
+  "Katanemo is licensed under the Katanemo Labs Community License.  
+  Copyright © Katanemo Labs, Inc. All Rights Reserved."
+
+  d. Compliance Requirement  
+  All use of the Katanemo Materials must comply with the Acceptable Use Policy, available at: https://katanemo.com/use-policy
+
+---
+
+4. COMMERCIAL USE AND DISTRIBUTION RESTRICTIONS
+
+You may use, reproduce, modify, distribute, and create derivative works from the Katanemo Materials for any purpose, including commercial use, EXCEPT in the following cases:
+
+You may NOT package, distribute, or make available the Katanemo Materials as part of:
+- A framework,
+- A proxy server,
+- Middleware,
+- A gateway infrastructure product,
+- Or any product substantially similar in function or purpose to the above,
+
+unless you obtain a separate commercial license from Katanemo Labs.
+
+---
+
+This license does not grant trademark rights or rights outside the scope described above.
+
+5. Disclaimer of Warranty  
+The Katanemo Materials are provided "AS IS" without warranties of any kind, either express or implied, including but not limited to warranties of title, non-infringement, or fitness for a particular purpose.
+
+---
+
+6. Limitation of Liability  
+Katanemo Labs is not liable for any indirect, special, or consequential damages arising out of the use of the Katanemo Materials, even if advised of the possibility of such damages.
+
+---
+
+7. Intellectual Property
+  a. Trademarks  
+  No trademark licenses are granted, except as required for attribution as described in Section 1.b. You may use the “Katanemo” mark according to Katanemo Labs' brand guidelines.
+
+  b. Ownership  
+  You own any derivative works or modifications you create, except for portions owned by Katanemo Labs.
+
+  c. Litigation  
+  If you file a lawsuit against Katanemo Labs regarding intellectual property, your license under this Agreement terminates.
+
+---
+
+8. Term and Termination  
+This Agreement continues until terminated. Katanemo Labs may terminate the Agreement if you breach any terms. Upon termination, you must cease using the Katanemo Materials.
+
+---
+
+10. Governing Law and Jurisdiction  
+This Agreement is governed by the laws of the State of Washington, USA. Any disputes will be resolved in the courts of California.

README.md

@@ -0,0 +1,125 @@
+---
+license: other
+license_name: katanemo-research
+license_link: >-
+  https://huggingface.co/katanemo/Plano-Orchestrator-30B-A3B/blob/main/LICENSE
+base_model:
+- Qwen/Qwen3-30B-A3B-Instruct-2507
+language:
+- en
+pipeline_tag: text-generation
+---
+# katanemo/Plano-Orchestrator-30B-A3B
+
+## Overview
+
+**Plano-Orchestrator** is a family of state-of-the-art routing and orchestration models that decide which agent(s) or LLM(s) should handle each request, and in what sequence. Built for multi-agent orchestration systems, Plano-Orchestrator excels at analyzing user intent and conversation context to make precise routing and orchestration decisions. Designed for real-world deployments, it delivers strong performance across general conversations, coding tasks, and long-context multi-turn conversations, while remaining efficient enough for low-latency production environments.
+
+#### Key capabilities
+- **Multi-turn Context Understanding**: Makes routing decisions based on full conversation history, maintaining contextual awareness across extended dialogues with evolving user needs.
+- **Multi-intent Detection**: Identifies when a single user message requires multiple agents simultaneously, enabling parallel/sequential routing to fulfill complex requests.
+- **Context-dependent Routing**: Correctly interprets ambiguous or referential messages by leveraging prior conversation context for accurate routing decisions.
+- **Conversational Flow Handling**: Understands diverse interaction patterns including follow-ups, clarifications, confirmations, and corrections within ongoing conversations.
+- **Negative Case Detection**: Recognizes when no specialized routing is needed, avoiding unnecessary LLM or agent calls for casual conversation.
+
+## Benchmark
+
+We evaluate on **1,958 user messages** across **605 multi-turn conversations** with more than **130 different agents**, covering three scenarios:
+
+- **General** (1,438 messages): Everyday conversational queries spanning diverse topics and agent types
+- **Coding** (285 messages): Development-focused conversations including debugging, code generation, and technical assistance
+- **Long-context** (235 messages): Extended conversations requiring understanding of extensive prior context
+
+ Each message is annotated with routing-relevant attributes, including not limited to intent multiplicity, context dependency, and continuation type. Below is the evaluation 
+ result.
+
+<div align="center">
+  <img width="100%" height="auto" src="./assets/Plano-Orchestrator.png"></a>
+</div>
+
+> [!NOTE]
+> For evaluation, please note that all models were evaluated with minimal reasoning to ensure routing remains efficient.
+
+## Example
+
+```python
+import json
+import torch
+
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+
+ORCHESTRATION_PROMPT = (
+    "You are a helpful assistant that selects the most suitable routes based on user intent.\n"
+    "You are provided with a list of available routes enclosed within <routes></routes> XML tags:\n"
+    "<routes>\n{routes}\n</routes>\n\n"
+    "You are also given the conversation context enclosed within <conversation></conversation> XML tags:\n"
+    "<conversation>\n{conversation}\n</conversation>\n\n"
+    "## Instructions\n"
+    "1. Analyze the latest user intent from the conversation.\n"
+    "2. Compare it against the available routes to find which routes can help fulfill the request.\n"
+    "3. Respond only with the exact route names from <routes>.\n"
+    "4. If no routes can help or the intent is already fulfilled, return an empty list.\n\n"
+    "## Response Format\n"
+    "Return your answer strictly in JSON as follows:\n"
+    '{{"route": ["route_name_1", "route_name_2", "..."]}}\n'
+    "If no routes are needed, return an empty list for `route`."
+)
+
+def convert_agents_to_routes(agents):
+    tools = [
+        {
+            "name": agent["name"],
+            "description": agent["description"],
+        }
+        for agent in agents
+    ]
+    return "\n".join([json.dumps(tool, ensure_ascii=False) for tool in tools])
+
+def build_messages(available_agents, conversation):
+    routes = convert_agents_to_routes(available_agents)
+    conversation_str = json.dumps(conversation, indent=4, ensure_ascii=False)
+    prompt = ORCHESTRATION_PROMPT.format(routes=routes, conversation=conversation_str)
+    return [{"role": "user", "content": prompt}]
+
+# Load model
+model_name = "katanemo/Plano-Orchestrator-30B-A3B"
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    torch_dtype=torch.float16,
+    device_map="auto"
+)
+
+# Define available agents
+available_agents = [
+    {"name": "WeatherAgent", "description": "Provides weather forecasts and current conditions for any location"},
+    {"name": "CodeAgent", "description": "Generates, debugs, explains, and reviews code in multiple programming languages"}
+]
+
+# Conversation history
+conversation = [
+    {"role": "user", "content": "What's the weather like today?"},
+    {"role": "assistant", "content": "I can help you with that. Could you tell me your location?"},
+    {"role": "user", "content": "San Francisco"},
+]
+
+# Build messages and generate
+model_inputs = tokenizer.apply_chat_template(
+    messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
+).to(model.device)
+
+generated_ids = model.generate(**model_inputs, max_new_tokens=32768)
+generated_ids = [
+    output_ids[len(input_ids) :]
+    for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
+]
+
+response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
+print(response)
+# Output: {"route": ["WeatherAgent"]}
+```
+
+## License
+
+The Plano-Orchestrator collection is distributed under the [Katanemo license](https://huggingface.co/katanemo/Plano-Orchestrator-30B-A3B/blob/main/LICENSE).

chat_template.jinja

@@ -0,0 +1,61 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if message.content is string %}
+        {%- set content = message.content %}
+    {%- else %}
+        {%- set content = '' %}
+    {%- endif %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

config.json

@@ -0,0 +1,38 @@
+{
+  "architectures": [
+    "Qwen3MoeForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 151643,
+  "decoder_sparse_step": 1,
+  "eos_token_id": 151645,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 2048,
+  "initializer_range": 0.02,
+  "intermediate_size": 6144,
+  "max_position_embeddings": 262144,
+  "max_window_layers": 48,
+  "mlp_only_layers": [],
+  "model_type": "qwen3_moe",
+  "moe_intermediate_size": 768,
+  "norm_topk_prob": true,
+  "num_attention_heads": 32,
+  "num_experts": 128,
+  "num_experts_per_tok": 8,
+  "num_hidden_layers": 48,
+  "num_key_value_heads": 4,
+  "output_router_logits": false,
+  "rms_norm_eps": 1e-06,
+  "rope_scaling": null,
+  "rope_theta": 10000000,
+  "router_aux_loss_coef": 0.001,
+  "sliding_window": null,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.52.4",
+  "use_cache": true,
+  "use_sliding_window": false,
+  "vocab_size": 151936
+}

generation_config.json

@@ -0,0 +1,13 @@
+{
+  "bos_token_id": 151643,
+  "do_sample": true,
+  "eos_token_id": [
+    151645,
+    151643
+  ],
+  "pad_token_id": 151643,
+  "temperature": 0.7,
+  "top_k": 20,
+  "top_p": 0.8,
+  "transformers_version": "4.52.4"
+}

model-00001-of-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8eee33dff0a547b3862f08e27f1e4944bee9ceccb6418e406491a33b780a0376
+size 4997184968

model-00002-of-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4e4f64e2f916bc6bc2a473ebe25a3ec8e71d5beaa2bfe896440752bff706d345
+size 4997741608

model-00003-of-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:102fec235d80fd8e1936813d53a8958dc08fe7127653833f38f0d5c17cea4b49
+size 4997742208

model-00004-of-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b65f7982f12c649f3160ebf7e5c0d26be501d269ebe035bc390b94a67639df2
+size 4997743184

model-00005-of-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0061945a1e1acf344bdb2677988a5d351ca236b1b42783ae9a23f39296b407ac
+size 4997743184

model-00006-of-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ef0af4bc624d6dbff9985cb1b1beb2d18d07c47763a911f618aef19bfe7e312c
+size 4997743184

model-00007-of-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0ef8c43b6fa66d7433e781e5292c22a408ade51bd98e2bf669d12eff6ef41ed1
+size 4997743184

model-00008-of-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a2565875f9da73bb91eedd5d9c22b232e7574635452737a8348ae1eb667cc30f
+size 4997743184

model-00009-of-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:608755ac8e441b798b2fe5b5eb1b29378a77c8c798ad6deee14894c653e68bc5
+size 4997743184

model-00010-of-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:937ac5288a8f6e06e7b5d62f42d867f64371231e3d2ad067ba3685f266b02f08
+size 4997743184

model-00011-of-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dcb296de73d5c386759cc728fc198c92cf5d93bcdfb78efee09daf3d95ee764e
+size 4997743184

model-00012-of-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b40966b93c7d12615c815463bb03ea4fb4abb58497538618353d279e6efd9bee
+size 4997743184

model-00013-of-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e18b553605082871839ca426e29ddb1cb149c1c0b0419b598cfbaba0417d21cf
+size 1094220288

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aeb13307a71acd8fe81861d94ad54ab689df773318809eed3cbe794b4492dae4
+size 11422654

tokenizer_config.json

@@ -0,0 +1,240 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151665": {
+      "content": "<tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151666": {
+      "content": "</tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151667": {
+      "content": "<think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151668": {
+      "content": "</think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 1010000,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "left",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}