Use https://github.com/lovedheart/llama.cpp/tree/ling_support to quantize. For model inference, please check out the branch and compile the code by yourself or wait until llama.cpp merges the PR https://github.com/ggml-org/llama.cpp/pull/16036

IQ3_XXS quantization details

--imatrix ling-mini-2.0-imatrix.gguf_dat
--leave-output-tensor
--token-embedding-type BF16
--tensor-type ffn_=BF16
--tensor-type ffn_down_exps=IQ4_NL --tensor-type blk.[1,5,19].ffn_down_exps=BF16
--tensor-type blk.[8,11,13,16,17,18].ffn_down_exps=Q6_K
--tensor-type ffn_gate_exps=IQ3_XXS --tensor-type blk.[1].ffn_gate_exps=Q6_K
--tensor-type blk.[7-11].ffn_gate_exps=IQ3_XXS
--tensor-type blk.[12-15].ffn_gate_exps=IQ4_XS --tensor-type blk.[16-19].ffn_gate_exps=Q6_K
--tensor-type ffn_up_exps=IQ3_XXS --tensor-type blk.[1].ffn_up_exps=Q6_K
--tensor-type blk.[7-11].ffn_up_exps=IQ3_XXS
--tensor-type blk.[12-15].ffn_up_exps=IQ4_XS --tensor-type blk.[16-19].ffn_up_exps=Q6_K
--tensor-type shexp=BF16

llama-cli test with 780M integrated GPU (ROCm 6.4.2)

M:\llama_ling>.\build\bin\llama-cli.exe -m N:\LLM\Ling-2.0\Ling-mini-2.0-IQ3_XXS.gguf
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon 780M Graphics, gfx1103 (0x1103), VMM: no, Wave Size: 32
build: 6471 (09e3df4b) with clang version 20.0.0git (git@github.com:Compute-Mirrors/llvm-project 33ab2c2f7838239f1e2e5c06432bbb8d887e8cb2) for x86_64-pc-windows-msvc
main: llama backend init
main: load the model and apply lora adapter, if any
llama_model_load_from_file_impl: using device ROCm0 (AMD Radeon 780M Graphics) (0000:c7:00.0) - 59175 MiB free
llama_model_loader: loaded meta data with 44 key-value pairs and 318 tensors from N:\LLM\Ling-2.0\Ling-mini-2.0-IQ3_XXS.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = bailingmoe-v2
llama_model_loader: - kv 1: general.type str = model
llama_model_loader: - kv 2: general.name str = Ling 2.0
llama_model_loader: - kv 3: general.version str = 2.0
llama_model_loader: - kv 4: general.basename str = Ling
llama_model_loader: - kv 5: general.size_label str = 256x1.0B
llama_model_loader: - kv 6: bailingmoe-v2.block_count u32 = 20
llama_model_loader: - kv 7: bailingmoe-v2.context_length u32 = 32768
llama_model_loader: - kv 8: bailingmoe-v2.embedding_length u32 = 2048
llama_model_loader: - kv 9: bailingmoe-v2.feed_forward_length u32 = 5120
llama_model_loader: - kv 10: bailingmoe-v2.attention.head_count u32 = 16
llama_model_loader: - kv 11: bailingmoe-v2.attention.head_count_kv u32 = 4
llama_model_loader: - kv 12: bailingmoe-v2.rope.freq_base f32 = 600000.000000
llama_model_loader: - kv 13: bailingmoe-v2.attention.layer_norm_rms_epsilon f32 = 0.000001
llama_model_loader: - kv 14: bailingmoe-v2.expert_used_count u32 = 8
llama_model_loader: - kv 15: bailingmoe-v2.attention.key_length u32 = 128
llama_model_loader: - kv 16: bailingmoe-v2.attention.value_length u32 = 128
llama_model_loader: - kv 17: bailingmoe-v2.rope.dimension_count u32 = 64
llama_model_loader: - kv 18: bailingmoe-v2.rope.scaling.type str = none
llama_model_loader: - kv 19: bailingmoe-v2.leading_dense_block_count u32 = 1
llama_model_loader: - kv 20: bailingmoe-v2.vocab_size u32 = 157184
llama_model_loader: - kv 21: bailingmoe-v2.expert_feed_forward_length u32 = 512
llama_model_loader: - kv 22: bailingmoe-v2.expert_weights_scale f32 = 2.500000
llama_model_loader: - kv 23: bailingmoe-v2.expert_count u32 = 256
llama_model_loader: - kv 24: bailingmoe-v2.expert_shared_count u32 = 1
llama_model_loader: - kv 25: bailingmoe-v2.expert_weights_norm bool = true
llama_model_loader: - kv 26: bailingmoe-v2.expert_gating_func u32 = 2
llama_model_loader: - kv 27: tokenizer.ggml.model str = gpt2
llama_model_loader: - kv 28: tokenizer.ggml.pre str = bailingmoe-v2
llama_model_loader: - kv 29: tokenizer.ggml.tokens arr[str,157184] = ["!", """, "#", "$", "%", "&", "'", ...
llama_model_loader: - kv 30: tokenizer.ggml.token_type arr[i32,157184] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv 31: tokenizer.ggml.merges arr[str,156635] = ["Ġ Ġ", "Ġ t", "i n", "Ġ a", "h e...
llama_model_loader: - kv 32: tokenizer.ggml.bos_token_id u32 = 156891
llama_model_loader: - kv 33: tokenizer.ggml.eos_token_id u32 = 156895
llama_model_loader: - kv 34: tokenizer.ggml.padding_token_id u32 = 156892
llama_model_loader: - kv 35: tokenizer.ggml.add_bos_token bool = false
llama_model_loader: - kv 36: tokenizer.ggml.add_eos_token bool = false
llama_model_loader: - kv 37: tokenizer.chat_template str = {% set thinking_option = 'off' %}\n{{-...
llama_model_loader: - kv 38: general.quantization_version u32 = 2
llama_model_loader: - kv 39: general.file_type u32 = 23
llama_model_loader: - kv 40: quantize.imatrix.file str = ling-mini-2.0-imatrix.gguf_dat
llama_model_loader: - kv 41: quantize.imatrix.dataset str = merged_calibration_all.txt
llama_model_loader: - kv 42: quantize.imatrix.entries_count u32 = 216
llama_model_loader: - kv 43: quantize.imatrix.chunks_count u32 = 1460
llama_model_loader: - type f32: 119 tensors
llama_model_loader: - type q6_K: 16 tensors
llama_model_loader: - type iq3_xxs: 20 tensors
llama_model_loader: - type iq4_nl: 10 tensors
llama_model_loader: - type iq4_xs: 8 tensors
llama_model_loader: - type bf16: 145 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type = IQ3_XXS - 3.0625 bpw
print_info: file size = 10.96 GiB (5.79 BPW)
load: printing all EOG tokens:
load: - 156892 ('<|endoftext|>')
load: - 156895 ('<|role_end|>')
load: special tokens cache size = 262
load: token to piece cache size = 1.0010 MB
print_info: arch = bailingmoe-v2
print_info: vocab_only = 0
print_info: n_ctx_train = 32768
print_info: n_embd = 2048
print_info: n_layer = 20
print_info: n_head = 16
print_info: n_head_kv = 4
print_info: n_rot = 64
print_info: n_swa = 0
print_info: is_swa_any = 0
print_info: n_embd_head_k = 128
print_info: n_embd_head_v = 128
print_info: n_gqa = 4
print_info: n_embd_k_gqa = 512
print_info: n_embd_v_gqa = 512
print_info: f_norm_eps = 0.0e+00
print_info: f_norm_rms_eps = 1.0e-06
print_info: f_clamp_kqv = 0.0e+00
print_info: f_max_alibi_bias = 0.0e+00
print_info: f_logit_scale = 0.0e+00
print_info: f_attn_scale = 0.0e+00
print_info: n_ff = 5120
print_info: n_expert = 256
print_info: n_expert_used = 8
print_info: causal attn = 1
print_info: pooling type = 0
print_info: rope type = 0
print_info: rope scaling = none
print_info: freq_base_train = 600000.0
print_info: freq_scale_train = 1
print_info: n_ctx_orig_yarn = 32768
print_info: rope_finetuned = unknown
print_info: model type = 16B
print_info: model params = 16.26 B
print_info: general.name = Ling 2.0
print_info: n_layer_dense_lead = 1
print_info: n_ff_exp = 512
print_info: n_expert_shared = 1
print_info: expert_weights_scale = 2.5
print_info: expert_weights_norm = 1
print_info: vocab type = BPE
print_info: n_vocab = 157184
print_info: n_merges = 156635
print_info: BOS token = 156891 '<|startoftext|>'
print_info: EOS token = 156895 '<|role_end|>'
print_info: EOT token = 156892 '<|endoftext|>'
print_info: PAD token = 156892 '<|endoftext|>'
print_info: LF token = 198 'Ċ'
print_info: EOG token = 156892 '<|endoftext|>'
print_info: EOG token = 156895 '<|role_end|>'
print_info: max token length = 154
load_tensors: loading model tensors, this can take a while... (mmap = true)
load_tensors: offloading 20 repeating layers to GPU
load_tensors: offloading output layer to GPU
load_tensors: offloaded 21/21 layers to GPU
load_tensors: ROCm0 model buffer size = 10610.36 MiB
load_tensors: CPU_Mapped model buffer size = 614.00 MiB
..............................................................
llama_context: constructing llama_context
llama_context: n_seq_max = 1
llama_context: n_ctx = 4096
llama_context: n_ctx_per_seq = 4096
llama_context: n_batch = 2048
llama_context: n_ubatch = 512
llama_context: causal_attn = 1
llama_context: flash_attn = auto
llama_context: kv_unified = false
llama_context: freq_base = 600000.0
llama_context: freq_scale = 1
llama_context: n_ctx_per_seq (4096) < n_ctx_train (32768) -- the full capacity of the model will not be utilized
llama_context: ROCm_Host output buffer size = 0.60 MiB
llama_kv_cache: ROCm0 KV buffer size = 160.00 MiB
llama_kv_cache: size = 160.00 MiB ( 4096 cells, 20 layers, 1/1 seqs), K (f16): 80.00 MiB, V (f16): 80.00 MiB
llama_context: Flash Attention was auto, set to enabled
llama_context: ROCm0 compute buffer size = 319.00 MiB
llama_context: ROCm_Host compute buffer size = 12.01 MiB
llama_context: graph nodes = 1353
llama_context: graph splits = 2
common_init_from_params: added <|endoftext|> logit bias = -inf
common_init_from_params: added <|role_end|> logit bias = -inf
common_init_from_params: setting dry_penalty_last_n to ctx_size = 4096
common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
main: llama threadpool init, n_threads = 8
main: chat template is available, enabling conversation mode (disable it with -no-cnv)
main: chat template example:
SYSTEMYou are a helpful assistantHUMANHelloASSISTANTHi thereHUMANHow are you?ASSISTANT

main: interactive mode on.
sampler seed: 959413357
sampler params:
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000 dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 4096
top_k = 40, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.800
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000
sampler chain: logits -> logit-bias -> penalties -> dry -> top-n-sigma -> top-k -> typical -> top-p -> min-p -> xtc -> temp-ext -> dist
generate: n_ctx = 4096, n_batch = 2048, n_predict = -1, n_keep = 0

== Running in interactive mode. ==

Press Ctrl+C to interject at any time.
Press Return to return control to the AI.
To return control without starting a new line, end your input with '/'.
If you want to submit another line, end your input with ''.
Not using system message. To change it, set a different value via -sys PROMPT

我有一个物理题。请你帮我解决。我有一个vehicle mass 质量为M kg，轮胎的半径为R，单位为米。axle之前有一个ratio为n的transmission。 n定义为input speed 除以output speed。求换算到transmission input端的vehicle inertia。请忽略轮胎质量。我们来一步步解决这个问题。

🧩 题目解析

你有一个质量为 $ M $ kg 的车辆，轮胎半径为 $ R $ 米。传动系统有一个 减速比（transmission ratio）为 $ n $，定义为：

$n = \frac{\text{input speed}}{\text{output speed}}$

也就是说，如果传动系统的输出轴（驱动轮）的转速为 $ \omega_{\text{out}} $，那么输入端的转速（发动机/传动轴）为：

$\omega_{\text{in}} = n \cdot \omega_{\text{out}}$

🎯 目标

求 换算到传动输入端的车辆惯性（vehicle inertia）。

注意：题目中说“忽略轮胎质量”，所以我们只考虑车辆整体的惯性矩，通常可以简化为绕某个轴的转动惯量。但题目没有给出具体的轴信息，我们假设是绕垂直轴（比如y轴）的转动惯量，通常车辆绕垂直轴的惯性可以近似为：

$I_{\text{vehicle}} = M \cdot r^2$

其中 $ r $ 是质心到旋转轴的垂直距离。但题目中没有给出这个信息，所以可能只是要求将车辆的转动惯量“换算”到输入端，考虑到传动比的转速变化带来的动能等效。

🧠 思路

我们考虑车辆的动能，它通常由两部分组成：

平动动能（translational）：$ \frac{1}{2} M v^2 $
转动动能（rotational）：$ \frac{1}{2} I \omega^2 $

但由于题目中只提到“vehicle inertia”，且忽略轮胎质量，我们可能只需要考虑车辆绕某个轴的转动惯量，并通过传动比将输出端的动能等效到输入端，从而换算出“惯性”。

🔁 传动系统中的动能等效

在传动系统中，动能是守恒的（忽略摩擦等损耗），所以：

$\frac{1}{2} I_{\text{out}} \omega_{\text{out}}^2 = \frac{1}{2} I_{\text{in}} \omega_{\text{in}}^2$

所以：

$I_{\text{in}} = I_{\text{out}} \cdot \left( \frac{\omega_{\text{out}}}{\omega_{\text{in}}} \right)^2 = I_{\text{out}} \cdot \left( \frac{1}{n} \right)^2$

因为 $ \omega_{\text{in}} = n \omega_{\text{out}} \Rightarrow \frac{\omega_{\text{out}}}{\omega_{\text{in}}} = \frac{1}{n} $

🚗 应用于车辆

如果我们将车辆绕垂直轴旋转（例如绕车轴），其转动惯量为 $ I_{\text{vehicle}} $，那么：

输出端的转动惯量（车轮）为 $ I_{\text{wheel}} $
输入端的等效转动惯量（发动机侧）为：

$I_{\text{in}} = I_{\text{wheel}} \cdot \left( \frac{1}{n} \right)^2$

但题目中只提到“vehicle inertia”，并没有提到车轮的惯性，也没有说明是绕哪个轴的。

✅ 最合理的解释（结合题意）

题目说：“忽略轮胎质量”，所以只考虑车辆整体的惯性。通常车辆绕垂直轴的惯性可以近似为：

$I_{\text{vehicle}} = M R^2$

这是假设车辆的质心在距离旋转轴 $ R $ 的位置（虽然不太合理，但可能是简化模型）。或者更合理的是，车辆绕其质心的惯性，但题目没有给出具体轴信息。

但如果我们考虑传动系统中，输出端的动能等于输入端的动能（忽略损耗），那么：

$\frac{1}{2} I_{\text{out}} \omega_{\text{out}}^2 = \frac{1}{2} I_{\text{in}} \omega_{\text{in}}^2 \Rightarrow I_{\text{in}} = I_{\text{out}} \cdot \left( \frac{\omega_{\text{out}}}{\omega_{\text{in}}} \right)^2 = I_{\text{out}} \cdot \left( \frac{1}{n} \right)^2$

所以，将输出端的惯性 $ I_{\text{out}} $ 等效到输入端时，要乘以 $ \frac{1}{n^2} $

🧮 结论

如果题目中的“vehicle inertia”指的是车辆绕某个轴的转动惯量，而我们要将它等效到传动输入端，那么：

$\boxed{ \frac{I}{n^2} }$

其中 $ I $ 是车辆绕该轴的转动惯量。

📌 如果题目中 vehicle inertia 是 $ I = M R^2 $（绕轴旋转）

那么换算到传动输入端的惯性为：

$\boxed{ \frac{M R^2}{n^2} }$

✅ 最终答案：

$\boxed{\frac{M R^2}{n^2}}$

这是将车辆绕某个轴（如轮胎旋转轴的垂直轴）的惯性等效到传动输入端后的结果。

如果你有更多上下文（比如是绕车轴、绕垂直轴、是否是纯滚动等），可以进一步调整。但基于现有信息，这是最合理的推导。

需要我帮你画图或进一步解释吗？🙂

需要你画图由于我目前是文字交互模式，无法直接画图，但我可以为你详细描述一个示意图，帮助你想象或自己绘制。

🚗 车辆传动系统示意图描述（文字版）

[发动机] --(输入轴)-- [传动系统] --(输出轴)-- [驱动轮]
                                      |
                                      ↓
                             [轮胎与地面接触]

传动系统有减速比 $ n $，即输入轴转速是输出轴的 $ n $ 倍。
输出轴连接驱动轮，驱动轮与地面接触。
车辆质量为 $ M $，轮胎半径为 $ R $。

🔄 动能等效图示（文字版）

输出端（驱动轮侧）：

驱动轮的角速度为 $ \omega_{\text{out}} $
驱动轮的转动惯量为 $ I_{\text{wheel}} $
驱动轮的动能：$ \frac{1}{2} I_{\text{wheel}} \omega_{\text{out}}^2 $

输入端（发动机侧）：

输入轴角速度为 $ \omega_{\text{in}} = n \omega_{\text{out}} $
发动机侧的等效转动惯量为 $ I_{\text{in}} $
发动机侧的动能：$ \frac{1}{2} I_{\text{in}} \omega_{\text{in}}^2 $

由于能量守恒（忽略损耗）：

$\frac{1}{2} I_{\text{wheel}} \omega_{\text{out}}^2 = \frac{1}{2} I_{\text{in}} (n \omega_{\text{out}})^2 \Rightarrow I_{\text{in}} = \frac{I_{\text{wheel}}}{n^2}$

🧮 如果只考虑车辆整体惯性（绕垂直轴）

假设车辆绕垂直轴（如y轴）旋转，其转动惯量为：

$I_{\text{vehicle}} = M R^2$

其中 $ R $ 是质心到旋转轴的距离（简化模型）。

将这个惯性“等效”到输入端（发动机侧），由于转速降低了 $ n $ 倍，所以动能相同：

$\frac{1}{2} I_{\text{vehicle}} \omega_{\text{in}}^2 = \frac{1}{2} I_{\text{equiv}} \omega_{\text{out}}^2 \Rightarrow I_{\text{equiv}} = I_{\text{vehicle}} \cdot \left( \frac{\omega_{\text{out}}}{\omega_{\text{in}}} \right)^2 = I_{\text{vehicle}} \cdot \frac{1}{n^2}$

所以：

$\boxed{ \frac{M R^2}{n^2} }$

🧾 总结

将车辆绕垂直轴的惯性 $ I = M R^2 $ 换算到传动输入端时，由于输入转速更高（$ \omega_{\text{in}} = n \omega_{\text{out}} $），所以等效惯性要除以 $ n^2 $。

如果你有具体的轴信息或更详细的物理模型，可以告诉我，我可以进一步调整推导。

需要我帮你把这个结果写成公式形式或 LaTeX 格式吗？比如：

$\boxed{I_{\text{in}} = \frac{M R^2}{n^2}}$

llama_perf_sampler_print: sampling time = 44.63 ms / 756 runs ( 0.06 ms per token, 16937.38 tokens per second)
llama_perf_context_print: load time = 6052.84 ms
llama_perf_context_print: prompt eval time = 581.36 ms / 82 tokens ( 7.09 ms per token, 141.05 tokens per second)
llama_perf_context_print: eval time = 52451.02 ms / 1946 runs ( 26.95 ms per token, 37.10 tokens per second)
llama_perf_context_print: total time = 113260.75 ms / 2028 tokens
llama_perf_context_print: graphs reused = 1939
Interrupted by user

Downloads last month: 20

GGUF

Model size

16B params

Architecture

bailingmoe-v2

Hardware compatibility

3-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for lovedheart/Ling-mini-2.0-GGUF

Base model

inclusionAI/Ling-mini-base-2.0

Finetuned

inclusionAI/Ling-mini-2.0

Quantized

(26)

this model