meta-models/Muse-Glimmer-30B-GGUF

Hardware
1× NVIDIA GeForce RTX 5090
Workload
256 → 1024
Runs
7 eligible
Efficient frontierOther eligible run
p95 TTFT (ms)Request throughput (req/s)0486.300run_ABIAlnoJQOVFipdw: 0.08 req/s, p95 TTFT 69.7 msrun_D8JKmhP0oZ4b10Yt: 0.08 req/s, p95 TTFT 69.3 msrun_r2hT1Fvin_tihPGr: 0.08 req/s, p95 TTFT 69.5 msrun_SRteyZHstMez-NPA: 0.14 req/s, p95 TTFT 486.3 msrun_Rv92orXGyCS2R1nc: 0.08 req/s, p95 TTFT 67.5 msrun_TSbZl1OFV0-VveGL: 0.08 req/s, p95 TTFT 67.1 msrun_3cKqnZbIlPZjdQvF: 0.08 req/s, p95 TTFT 67.2 ms

Runs

All published runs with the same model build, accelerator setup, and request shape. Benchmark methodology remains attached to each run.

RuntimeMaterial parametersTest loadReq/sOutput tok/sp95 TTFTp95 TPOTp95 E2EFailuresRun
llama.cpp b10689ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=182.469.7 ms12.09 ms12,439.2 ms0run_ABIAlnoJQOVFipdw
llama.cpp b10689ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=182.469.3 ms12.1 ms12,441.2 ms0run_D8JKmhP0oZ4b10Yt
llama.cpp b10689ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=182.369.5 ms12.13 ms12,468.6 ms0run_r2hT1Fvin_tihPGr
llama.cpp b10689ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=8147.5486.3 ms54.57 ms56,133.1 ms0run_SRteyZHstMez-NPA
llama.cpp b10689ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=182.267.5 ms12.13 ms12,470.7 ms0run_Rv92orXGyCS2R1nc
llama.cpp b10689ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=182.267.1 ms12.13 ms12,474.6 ms0run_TSbZl1OFV0-VveGL
llama.cpp b10689ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=182.267.2 ms12.15 ms12,489.6 ms0run_3cKqnZbIlPZjdQvF
Runtime
llama.cpp b10689
Material parameters
ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=1
Req/s
Output tok/s
82.4
p95 TTFT / TPOT
69.7 ms / 12.09 ms
Runtime
llama.cpp b10689
Material parameters
ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=1
Req/s
Output tok/s
82.4
p95 TTFT / TPOT
69.3 ms / 12.1 ms
Runtime
llama.cpp b10689
Material parameters
ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=1
Req/s
Output tok/s
82.3
p95 TTFT / TPOT
69.5 ms / 12.13 ms
Runtime
llama.cpp b10689
Material parameters
ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=8
Req/s
Output tok/s
147.5
p95 TTFT / TPOT
486.3 ms / 54.57 ms
Runtime
llama.cpp b10689
Material parameters
ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=1
Req/s
Output tok/s
82.2
p95 TTFT / TPOT
67.5 ms / 12.13 ms
Runtime
llama.cpp b10689
Material parameters
ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=1
Req/s
Output tok/s
82.2
p95 TTFT / TPOT
67.1 ms / 12.13 ms
Runtime
llama.cpp b10689
Material parameters
ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=1
Req/s
Output tok/s
82.2
p95 TTFT / TPOT
67.2 ms / 12.15 ms