error on long context beyond 58k+ tokens

by mtcl - opened Sep 25

mtcl

Sep 25

(base) mukul@jarvis:/dev-ai/llama.cpp$ CUDA_DEVICE_ORDER=PCI_BUS_ID CUDA_VISIBLE_DEVICES="0,1,2,3" ./build/bin/llama-server
--model /media/mukul/t7/models/unsloth/DeepSeek-V3.1-Terminus-GGUF/UD-Q2_K_XL/DeepSeek-V3.1-Terminus-UD-Q2_K_XL-00001-of-00006.gguf
--alias unsloth/DeepSeek-V3.1-Terminus
--ctx-size 163840
--temp 0.6
--top_p 0.95
--min_p 0.01
-fa on
-b 8192 -ub 8192
--n-gpu-layers 63
-ot exps=CPU
--jinja
--parallel 1
--threads 56
--host 0.0.0.0
--port 10002
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 4 CUDA devices:
Device 0: NVIDIA GeForce RTX 5090, compute capability 12.0, VMM: yes
Device 1: NVIDIA GeForce RTX 5090, compute capability 12.0, VMM: yes
Device 2: NVIDIA GeForce RTX 5090, compute capability 12.0, VMM: yes
Device 3: NVIDIA GeForce RTX 5090, compute capability 12.0, VMM: yes
build: 6582 (aa3ee0eb) with cc (Ubuntu 13.3.0-6ubuntu224.04) 13.3.0 for x86_64-linux-gnu
system info: n_threads = 56, n_threads_batch = 56, total_threads = 112

main: binding port with default address family
main: HTTP server is listening, hostname: 0.0.0.0, port: 10002, http threads: 111
main: loading model
srv load_model: loading model '/media/mukul/t7/models/unsloth/DeepSeek-V3.1-Terminus-GGUF/UD-Q2_K_XL/DeepSeek-V3.1-Terminus-UD-Q2_K_XL-00001-of-00006.gguf'
llama_model_load_from_file_impl: using device CUDA0 (NVIDIA GeForce RTX 5090) (0000:16:00.0) - 31586 MiB free
llama_model_load_from_file_impl: using device CUDA1 (NVIDIA GeForce RTX 5090) (0000:40:00.0) - 31585 MiB free
llama_model_load_from_file_impl: using device CUDA2 (NVIDIA GeForce RTX 5090) (0000:6a:00.0) - 30950 MiB free
llama_model_load_from_file_impl: using device CUDA3 (NVIDIA GeForce RTX 5090) (0000:94:00.0) - 31585 MiB free
llama_model_loader: additional 5 GGUFs metadata loaded.
llama_model_loader: loaded meta data with 61 key-value pairs and 1086 tensors from /media/mukul/t7/models/unsloth/DeepSeek-V3.1-Terminus-GGUF/UD-Q2_K_XL/DeepSeek-V3.1-Terminus-UD-Q2_K_XL-00001-of-00006.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = deepseek2
llama_model_loader: - kv 1: general.type str = model
llama_model_loader: - kv 2: general.name str = Deepseek-V3.1-Terminus
llama_model_loader: - kv 3: general.basename str = Deepseek-V3.1-Terminus
llama_model_loader: - kv 4: general.quantized_by str = Unsloth
llama_model_loader: - kv 5: general.size_label str = 256x20B
llama_model_loader: - kv 6: general.license str = mit
llama_model_loader: - kv 7: general.repo_url str = https://huggingface.co/unsloth
llama_model_loader: - kv 8: general.base_model.count u32 = 1
llama_model_loader: - kv 9: general.base_model.0.name str = DeepSeek V3.1 Terminus
llama_model_loader: - kv 10: general.base_model.0.organization str = Deepseek Ai
llama_model_loader: - kv 11: general.base_model.0.repo_url str = https://huggingface.co/deepseek-ai/De...
llama_model_loader: - kv 12: deepseek2.block_count u32 = 61
llama_model_loader: - kv 13: deepseek2.context_length u32 = 163840
llama_model_loader: - kv 14: deepseek2.embedding_length u32 = 7168
llama_model_loader: - kv 15: deepseek2.feed_forward_length u32 = 18432
llama_model_loader: - kv 16: deepseek2.attention.head_count u32 = 128
llama_model_loader: - kv 17: deepseek2.attention.head_count_kv u32 = 1
llama_model_loader: - kv 18: deepseek2.rope.freq_base f32 = 10000.000000
llama_model_loader: - kv 19: deepseek2.attention.layer_norm_rms_epsilon f32 = 0.000001
llama_model_loader: - kv 20: deepseek2.expert_used_count u32 = 8
llama_model_loader: - kv 21: deepseek2.leading_dense_block_count u32 = 3
llama_model_loader: - kv 22: deepseek2.vocab_size u32 = 129280
llama_model_loader: - kv 23: deepseek2.attention.q_lora_rank u32 = 1536
llama_model_loader: - kv 24: deepseek2.attention.kv_lora_rank u32 = 512
llama_model_loader: - kv 25: deepseek2.attention.key_length u32 = 576
llama_model_loader: - kv 26: deepseek2.attention.value_length u32 = 512
llama_model_loader: - kv 27: deepseek2.attention.key_length_mla u32 = 192
llama_model_loader: - kv 28: deepseek2.attention.value_length_mla u32 = 128
llama_model_loader: - kv 29: deepseek2.expert_feed_forward_length u32 = 2048
llama_model_loader: - kv 30: deepseek2.expert_count u32 = 256
llama_model_loader: - kv 31: deepseek2.expert_shared_count u32 = 1
llama_model_loader: - kv 32: deepseek2.expert_weights_scale f32 = 2.500000
llama_model_loader: - kv 33: deepseek2.expert_weights_norm bool = true
llama_model_loader: - kv 34: deepseek2.expert_gating_func u32 = 2
llama_model_loader: - kv 35: deepseek2.rope.dimension_count u32 = 64
llama_model_loader: - kv 36: deepseek2.rope.scaling.type str = yarn
llama_model_loader: - kv 37: deepseek2.rope.scaling.factor f32 = 40.000000
llama_model_loader: - kv 38: deepseek2.rope.scaling.original_context_length u32 = 4096
llama_model_loader: - kv 39: deepseek2.rope.scaling.yarn_log_multiplier f32 = 0.100000
llama_model_loader: - kv 40: tokenizer.ggml.model str = gpt2
llama_model_loader: - kv 41: tokenizer.ggml.pre str = deepseek-v3
llama_model_loader: - kv 42: tokenizer.ggml.tokens arr[str,129280] = ["<｜begin▁of▁sentence｜>", "<�...
llama_model_loader: - kv 43: tokenizer.ggml.token_type arr[i32,129280] = [3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv 44: tokenizer.ggml.merges arr[str,127741] = ["Ġ t", "Ġ a", "i n", "Ġ Ġ", "h e...
llama_model_loader: - kv 45: tokenizer.ggml.bos_token_id u32 = 0
llama_model_loader: - kv 46: tokenizer.ggml.eos_token_id u32 = 1
llama_model_loader: - kv 47: tokenizer.ggml.padding_token_id u32 = 2
llama_model_loader: - kv 48: tokenizer.ggml.add_bos_token bool = true
llama_model_loader: - kv 49: tokenizer.ggml.add_sep_token bool = false
llama_model_loader: - kv 50: tokenizer.ggml.add_eos_token bool = false
llama_model_loader: - kv 51: tokenizer.chat_template str = {#- Unsloth template fixes #}{% if no...
llama_model_loader: - kv 52: general.quantization_version u32 = 2
llama_model_loader: - kv 53: general.file_type u32 = 10
llama_model_loader: - kv 54: quantize.imatrix.file str = DeepSeek-V3.1-Terminus-GGUF/imatrix_u...
llama_model_loader: - kv 55: quantize.imatrix.dataset str = unsloth_calibration_DeepSeek-V3.1-Ter...
llama_model_loader: - kv 56: quantize.imatrix.entries_count u32 = 781
llama_model_loader: - kv 57: quantize.imatrix.chunks_count u32 = 84
llama_model_loader: - kv 58: split.no u16 = 0
llama_model_loader: - kv 59: split.tensors.count i32 = 1086
llama_model_loader: - kv 60: split.count u16 = 6
llama_model_loader: - type f32: 361 tensors
llama_model_loader: - type q8_0: 191 tensors
llama_model_loader: - type q2_K: 116 tensors
llama_model_loader: - type q3_K: 51 tensors
llama_model_loader: - type q4_K: 304 tensors
llama_model_loader: - type q5_K: 52 tensors
llama_model_loader: - type q6_K: 11 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type = Q2_K - Medium
print_info: file size = 234.06 GiB (3.00 BPW)
load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
load: printing all EOG tokens:
load: - 1 ('<｜end▁of▁sentence｜>')
load: special tokens cache size = 818
load: token to piece cache size = 0.8223 MB
print_info: arch = deepseek2
print_info: vocab_only = 0
print_info: n_ctx_train = 163840
print_info: n_embd = 7168
print_info: n_layer = 61
print_info: n_head = 128
print_info: n_head_kv = 1
print_info: n_rot = 64
print_info: n_swa = 0
print_info: is_swa_any = 0
print_info: n_embd_head_k = 576
print_info: n_embd_head_v = 512
print_info: n_gqa = 128
print_info: n_embd_k_gqa = 576
print_info: n_embd_v_gqa = 512
print_info: f_norm_eps = 0.0e+00
print_info: f_norm_rms_eps = 1.0e-06
print_info: f_clamp_kqv = 0.0e+00
print_info: f_max_alibi_bias = 0.0e+00
print_info: f_logit_scale = 0.0e+00
print_info: f_attn_scale = 0.0e+00
print_info: n_ff = 18432
print_info: n_expert = 256
print_info: n_expert_used = 8
print_info: causal attn = 1
print_info: pooling type = 0
print_info: rope type = 0
print_info: rope scaling = yarn
print_info: freq_base_train = 10000.0
print_info: freq_scale_train = 0.025
print_info: n_ctx_orig_yarn = 4096
print_info: rope_finetuned = unknown
print_info: model type = 671B
print_info: model params = 671.03 B
print_info: general.name = Deepseek-V3.1-Terminus
print_info: n_layer_dense_lead = 3
print_info: n_lora_q = 1536
print_info: n_lora_kv = 512
print_info: n_embd_head_k_mla = 192
print_info: n_embd_head_v_mla = 128
print_info: n_ff_exp = 2048
print_info: n_expert_shared = 1
print_info: expert_weights_scale = 2.5
print_info: expert_weights_norm = 1
print_info: expert_gating_func = sigmoid
print_info: rope_yarn_log_mul = 0.1000
print_info: vocab type = BPE
print_info: n_vocab = 129280
print_info: n_merges = 127741
print_info: BOS token = 0 '<｜begin▁of▁sentence｜>'
print_info: EOS token = 1 '<｜end▁of▁sentence｜>'
print_info: EOT token = 1 '<｜end▁of▁sentence｜>'
print_info: PAD token = 2 '<｜▁pad▁｜>'
print_info: LF token = 201 'Ċ'
print_info: FIM PRE token = 128801 '<｜fim▁begin｜>'
print_info: FIM SUF token = 128800 '<｜fim▁hole｜>'
print_info: FIM MID token = 128802 '<｜fim▁end｜>'
print_info: EOG token = 1 '<｜end▁of▁sentence｜>'
print_info: max token length = 256
load_tensors: loading model tensors, this can take a while... (mmap = true)
load_tensors: offloading 61 repeating layers to GPU
load_tensors: offloading output layer to GPU
load_tensors: offloaded 62/62 layers to GPU
load_tensors: CUDA0 model buffer size = 2854.32 MiB
load_tensors: CUDA1 model buffer size = 2288.18 MiB
load_tensors: CUDA2 model buffer size = 2179.10 MiB
load_tensors: CUDA3 model buffer size = 2793.48 MiB
load_tensors: CPU_Mapped model buffer size = 46821.99 MiB
load_tensors: CPU_Mapped model buffer size = 47111.91 MiB
load_tensors: CPU_Mapped model buffer size = 46061.48 MiB
load_tensors: CPU_Mapped model buffer size = 46885.06 MiB
load_tensors: CPU_Mapped model buffer size = 47273.39 MiB
load_tensors: CPU_Mapped model buffer size = 4394.39 MiB
....................................................................................................
llama_context: constructing llama_context
llama_context: n_seq_max = 1
llama_context: n_ctx = 163840
llama_context: n_ctx_per_seq = 163840
llama_context: n_batch = 8192
llama_context: n_ubatch = 8192
llama_context: causal_attn = 1
llama_context: flash_attn = enabled
llama_context: kv_unified = false
llama_context: freq_base = 10000.0
llama_context: freq_scale = 0.025
llama_context: CUDA_Host output buffer size = 0.49 MiB
llama_kv_cache: CUDA0 KV buffer size = 5440.00 MiB
llama_kv_cache: CUDA1 KV buffer size = 5440.00 MiB
llama_kv_cache: CUDA2 KV buffer size = 5100.00 MiB
llama_kv_cache: CUDA3 KV buffer size = 4760.00 MiB
llama_kv_cache: size = 20740.00 MiB (163840 cells, 61 layers, 1/1 seqs), K (f16): 10980.00 MiB, V (f16): 9760.00 MiB
llama_context: CUDA0 compute buffer size = 10992.16 MiB
llama_context: CUDA1 compute buffer size = 8608.16 MiB
llama_context: CUDA2 compute buffer size = 8608.16 MiB
llama_context: CUDA3 compute buffer size = 8608.19 MiB
llama_context: CUDA_Host compute buffer size = 5344.22 MiB
llama_context: graph nodes = 4785
llama_context: graph splits = 224 (with bs=8192), 121 (with bs=1)
common_init_from_params: added <｜end▁of▁sentence｜> logit bias = -inf
common_init_from_params: setting dry_penalty_last_n to ctx_size = 163840
common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
srv init: initializing slots, n_slots = 1
slot init: id 0 | task -1 | new slot n_ctx_slot = 163840
srv init: Enable thinking? 1
main: model loaded
main: chat template, chat_template: {#- Unsloth template fixes #}{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% if enable_thinking is defined and enable_thinking is false %}{% set thinking = false %}{% elif enable_thinking is defined and enable_thinking is true %}{% set thinking = true %}{% elif not thinking is defined %}{% set thinking = false %}{% endif %}{% set ns = namespace(is_first=false, is_tool=false, system_prompt='', is_first_sp=true, is_last_user=false) %}{%- for message in messages %}{%- if message['role'] == 'system' %}{%- if ns.is_first_sp %}{% set ns.system_prompt = ns.system_prompt + message['content'] %}{% set ns.is_first_sp = false %}{%- else %}{% set ns.system_prompt = ns.system_prompt + '

' + message['content'] %}{%- endif %}{%- endif %}{%- endfor %}{{ bos_token }}{{ ns.system_prompt }}{%- for message in messages %}{%- if message['role'] == 'user' %}{%- set ns.is_tool = false -%}{%- set ns.is_first = false -%}{%- set ns.is_last_user = true -%}{{'<｜User｜>' + message['content']}}{%- endif %}{%- if message['role'] == 'assistant' and message['tool_calls'] is defined and message['tool_calls'] is not none %}{%- if ns.is_last_user %}{{'<｜Assistant｜>'}}{%- endif %}{%- set ns.is_last_user = false -%}{%- set ns.is_first = false %}{%- set ns.is_tool = false -%}{%- for tool in message['tool_calls'] %}{%- if not ns.is_first %}{%- if message['content'] is none %}{{'<｜tool▁calls▁begin｜><｜tool▁call▁begin｜>'+ tool['function']['name'] + '<｜tool▁sep｜>' + tool['function']['arguments'] + '<｜tool▁call▁end｜>'}}{%- else %}{{message['content'] + '<｜tool▁calls▁begin｜><｜tool▁call▁begin｜>' + tool['function']['name'] + '<｜tool▁sep｜>' + tool['function']['arguments'] + '<｜tool▁call▁end｜>'}}{%- endif %}{%- set ns.is_first = true -%}{%- else %}{{'<｜tool▁call▁begin｜>'+ tool['function']['name'] + '<｜tool▁sep｜>' + tool['function']['arguments'] + '<｜tool▁call▁end｜>'}}{%- endif %}{%- endfor %}{{'<｜tool▁calls▁end｜><｜end▁of▁sentence｜>'}}{%- endif %}{%- if message['role'] == 'assistant' and (message['tool_calls'] is not defined or message['tool_calls'] is none) %}{%- if ns.is_last_user %}{{'<｜Assistant｜>'}}{%- if message['prefix'] is defined and message['prefix'] and thinking %}{{''}} {%- else %}{{''}}{%- endif %}{%- endif %}{%- set ns.is_last_user = false -%}{%- if ns.is_tool %}{{message['content'] + '<｜end▁of▁sentence｜>'}}{%- set ns.is_tool = false -%}{%- else %}{%- set content = message['content'] -%}{%- if '' in content %}{%- set splitted = content.split('') -%}{%- set content = splitted[1:] | join('') -%}{%- endif %}{{content + '<｜end▁of▁sentence｜>'}}{%- endif %}{%- endif %}{%- if message['role'] == 'tool' %}{%- set ns.is_last_user = false -%}{%- set ns.is_tool = true -%}{{'<｜tool▁output▁begin｜>' + message['content'] + '<｜tool▁output▁end｜>'}}{%- endif %}{%- endfor -%}{%- if add_generation_prompt and ns.is_last_user and not ns.is_tool %}{{'<｜Assistant｜>'}}{%- if not thinking %}{{''}}{%- else %}{{''}}{%- endif %}{% endif %}{#- Copyright 2025-present Unsloth. Apache 2.0 License. #}, example_format: 'You are a helpful assistant<｜User｜>Hello<｜Assistant｜>Hi there<｜end▁of▁sentence｜><｜User｜>How are you?<｜Assistant｜>'
main: server is listening on http://0.0.0.0:10002 - starting the main loop
srv update_slots: all slots are idle
srv log_server_r: request: GET /v1/models 172.17.0.3 200
srv log_server_r: request: GET /v1/models 172.17.0.3 200
srv log_server_r: request: GET /v1/models 172.17.0.3 200
srv params_from_: Chat format: DeepSeek V3.1
slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = -1
slot launch_slot_: id 0 | task 0 | processing task
slot update_slots: id 0 | task 0 | new prompt, n_ctx_slot = 163840, n_keep = 0, n_prompt_tokens = 5
slot update_slots: id 0 | task 0 | kv cache rm [0, end)
slot update_slots: id 0 | task 0 | prompt processing progress, n_past = 5, n_tokens = 5, progress = 1.000000
slot update_slots: id 0 | task 0 | prompt done, n_past = 5, n_tokens = 5
slot release: id 0 | task 0 | stop processing: n_past = 136, truncated = 0
slot print_timing: id 0 | task 0 |
prompt eval time = 377.53 ms / 5 tokens ( 75.51 ms per token, 13.24 tokens per second)
eval time = 10583.29 ms / 132 tokens ( 80.18 ms per token, 12.47 tokens per second)
total time = 10960.82 ms / 137 tokens
srv update_slots: all slots are idle
srv log_server_r: request: POST /v1/chat/completions 172.17.0.3 200
srv log_server_r: request: GET /v1/models 172.17.0.3 200
srv log_server_r: request: GET /v1/models 172.17.0.3 200
srv log_server_r: request: GET /v1/models 172.17.0.3 200
srv log_server_r: request: GET /v1/models 172.17.0.3 200
srv params_from_: Chat format: DeepSeek V3.1
slot get_availabl: id 0 | task 0 | selected slot by LRU, t_last = 1985407009394
slot launch_slot_: id 0 | task 133 | processing task
slot update_slots: id 0 | task 133 | new prompt, n_ctx_slot = 163840, n_keep = 0, n_prompt_tokens = 111221
slot update_slots: id 0 | task 133 | kv cache rm [1, end)
slot update_slots: id 0 | task 133 | prompt processing progress, n_past = 8193, n_tokens = 8192, progress = 0.073655
slot update_slots: id 0 | task 133 | kv cache rm [8193, end)
slot update_slots: id 0 | task 133 | prompt processing progress, n_past = 16385, n_tokens = 8192, progress = 0.147310
slot update_slots: id 0 | task 133 | kv cache rm [16385, end)
slot update_slots: id 0 | task 133 | prompt processing progress, n_past = 24577, n_tokens = 8192, progress = 0.220965
slot update_slots: id 0 | task 133 | kv cache rm [24577, end)
slot update_slots: id 0 | task 133 | prompt processing progress, n_past = 32769, n_tokens = 8192, progress = 0.294621
slot update_slots: id 0 | task 133 | kv cache rm [32769, end)
slot update_slots: id 0 | task 133 | prompt processing progress, n_past = 40961, n_tokens = 8192, progress = 0.368276
slot update_slots: id 0 | task 133 | kv cache rm [40961, end)
slot update_slots: id 0 | task 133 | prompt processing progress, n_past = 49153, n_tokens = 8192, progress = 0.441931
slot update_slots: id 0 | task 133 | kv cache rm [49153, end)
slot update_slots: id 0 | task 133 | prompt processing progress, n_past = 57345, n_tokens = 8192, progress = 0.515586
slot update_slots: id 0 | task 133 | kv cache rm [57345, end)
slot update_slots: id 0 | task 133 | prompt processing progress, n_past = 65537, n_tokens = 8192, progress = 0.589241
/home/mukul/dev-ai/llama.cpp/ggml/src/ggml-cuda/cpy.cu:284: GGML_ASSERT(ggml_nbytes(src0) <= INT_MAX) failed
[New LWP 3807786]
[New LWP 3807785]
[New LWP 3807784]
[New LWP 3807783]
[New LWP 3807782]
[New LWP 3807781]
[New LWP 3807780]
[New LWP 3807779]
[New LWP 3807778]
[New LWP 3807777]
[New LWP 3807776]
[New LWP 3807775]
[New LWP 3807774]
[New LWP 3807773]
[New LWP 3807772]
[New LWP 3807771]
[New LWP 3807770]
[New LWP 3807769]
[New LWP 3807768]
[New LWP 3807767]
[New LWP 3807766]
[New LWP 3807765]
[New LWP 3807764]
[New LWP 3807763]
[New LWP 3807762]
[New LWP 3807761]
[New LWP 3807760]
[New LWP 3807759]
[New LWP 3807758]
[New LWP 3807757]
[New LWP 3807756]
[New LWP 3807755]
[New LWP 3807754]
[New LWP 3807753]
[New LWP 3807752]
[New LWP 3807751]
[New LWP 3807750]
[New LWP 3807749]
[New LWP 3807748]
[New LWP 3807747]
[New LWP 3807746]
[New LWP 3807745]
[New LWP 3807744]
[New LWP 3807743]
[New LWP 3807742]
[New LWP 3807741]
[New LWP 3807740]
[New LWP 3807739]
[New LWP 3807738]
[New LWP 3807737]
[New LWP 3807736]
[New LWP 3807735]
[New LWP 3807734]
[New LWP 3807733]
[New LWP 3807732]
[New LWP 3797916]
[New LWP 3797915]
[New LWP 3797914]
[New LWP 3797913]
[New LWP 3797912]
[New LWP 3797911]
[New LWP 3797910]
[New LWP 3797909]
[New LWP 3797908]
[New LWP 3797907]
[New LWP 3797906]
[New LWP 3797905]
[New LWP 3797904]
[New LWP 3797903]
[New LWP 3797902]
[New LWP 3797901]
[New LWP 3797900]
[New LWP 3797899]
[New LWP 3797898]
[New LWP 3797897]
[New LWP 3797896]
[New LWP 3797895]
[New LWP 3797894]
[New LWP 3797893]
[New LWP 3797892]
[New LWP 3797891]
[New LWP 3797890]
[New LWP 3797889]
[New LWP 3797888]
[New LWP 3797887]
[New LWP 3797886]
[New LWP 3797885]
[New LWP 3797884]
[New LWP 3797883]
[New LWP 3797882]
[New LWP 3797881]
[New LWP 3797880]
[New LWP 3797879]
[New LWP 3797878]
[New LWP 3797877]
[New LWP 3797876]
[New LWP 3797875]
[New LWP 3797874]
[New LWP 3797873]
[New LWP 3797872]
[New LWP 3797871]
[New LWP 3797870]
[New LWP 3797869]
[New LWP 3797868]
[New LWP 3797867]
[New LWP 3797866]
[New LWP 3797865]
[New LWP 3797864]
[New LWP 3797863]
[New LWP 3797862]
[New LWP 3797861]
[New LWP 3797860]
[New LWP 3797859]
[New LWP 3797858]
[New LWP 3797857]
[New LWP 3797856]
[New LWP 3797855]
[New LWP 3797854]
[New LWP 3797853]
[New LWP 3797852]
[New LWP 3797851]
[New LWP 3797850]
[New LWP 3797849]
[New LWP 3797848]
[New LWP 3797847]
[New LWP 3797846]
[New LWP 3797845]
[New LWP 3797844]
[New LWP 3797843]
[New LWP 3797842]
[New LWP 3797841]
[New LWP 3797840]
[New LWP 3797839]
[New LWP 3797838]
[New LWP 3797837]
[New LWP 3797836]
[New LWP 3797835]
[New LWP 3797834]
[New LWP 3797833]
[New LWP 3797832]
[New LWP 3797831]
[New LWP 3797830]
[New LWP 3797829]
[New LWP 3797828]
[New LWP 3797827]
[New LWP 3797826]
[New LWP 3797825]
[New LWP 3797824]
[New LWP 3797823]
[New LWP 3797822]
[New LWP 3797821]
[New LWP 3797820]
[New LWP 3797819]
[New LWP 3797818]
[New LWP 3797817]
[New LWP 3797816]
[New LWP 3797815]
[New LWP 3797814]
[New LWP 3797813]
[New LWP 3797812]
[New LWP 3797811]
[New LWP 3797810]
[New LWP 3797809]
[New LWP 3797808]
[New LWP 3797807]
[New LWP 3797806]
[New LWP 3797805]
[New LWP 3797804]
[New LWP 3797803]
[New LWP 3797802]
[New LWP 3797800]
[New LWP 3797799]
[New LWP 3797798]
[New LWP 3797797]
[New LWP 3797789]
[New LWP 3797788]
[New LWP 3797780]

This GDB supports auto-downloading debuginfo from the following URLs:
https://debuginfod.ubuntu.com
Enable debuginfod for this session? (y or [n]) [answered N; input not from terminal]
Debuginfod has been disabled.
To make this setting permanent, add 'set debuginfod enabled off' to .gdbinit.
warning: could not find '.gnu_debugaltlink' file for /lib/x86_64-linux-gnu/liblber.so.2
warning: could not find '.gnu_debugaltlink' file for /lib/x86_64-linux-gnu/libbrotlidec.so.1
warning: could not find '.gnu_debugaltlink' file for /lib/x86_64-linux-gnu/libbrotlicommon.so.1
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007417ab510813 in __GI___wait4 (pid=3826419, stat_loc=0x0, options=0, usage=0x0) at ../sysdeps/unix/sysv/linux/wait4.c:30
warning: 30 ../sysdeps/unix/sysv/linux/wait4.c: No such file or directory
#0 0x00007417ab510813 in __GI___wait4 (pid=3826419, stat_loc=0x0, options=0, usage=0x0) at ../sysdeps/unix/sysv/linux/wait4.c:30
30 in ../sysdeps/unix/sysv/linux/wait4.c
#1 0x00007417abf22613 in ggml_print_backtrace () from /home/mukul/dev-ai/llama.cpp/build/bin/libggml-base.so
#2 0x00007417abf227bb in ggml_abort () from /home/mukul/dev-ai/llama.cpp/build/bin/libggml-base.so
#3 0x00007417a90fcf95 in ggml_cuda_cpy(ggml_backend_cuda_context&, ggml_tensor const*, ggml_tensor*, bool) () from /home/mukul/dev-ai/llama.cpp/build/bin/libggml-cuda.so
#4 0x00007417a91547c9 in ggml_backend_cuda_graph_compute(ggml_backend*, ggml_cgraph*) () from /home/mukul/dev-ai/llama.cpp/build/bin/libggml-cuda.so
#5 0x00007417abf3d667 in ggml_backend_sched_graph_compute_async () from /home/mukul/dev-ai/llama.cpp/build/bin/libggml-base.so
#6 0x00007417abca1131 in llama_context::graph_compute(ggml_cgraph*, bool) () from /home/mukul/dev-ai/llama.cpp/build/bin/libllama.so
#7 0x00007417abca2b54 in llama_context::process_ubatch(llama_ubatch const&, llm_graph_type, llama_memory_context_i*, ggml_status&) () from /home/mukul/dev-ai/llama.cpp/build/bin/libllama.so
#8 0x00007417abca8dff in llama_context::decode(llama_batch const&) () from /home/mukul/dev-ai/llama.cpp/build/bin/libllama.so
#9 0x00007417abca9d1f in llama_decode () from /home/mukul/dev-ai/llama.cpp/build/bin/libllama.so
#10 0x0000572f41ad51c2 in server_context::update_slots() ()
#11 0x0000572f41a9af48 in server_queue::start_loop() ()
#12 0x0000572f41a5ec15 in main ()
[Inferior 1 (process 3797761) detached]
Aborted (core dumped)
(base) mukul@jarvis:~/dev-ai/llama.cpp$

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment