meta-models/Muse-Glimmer-30B-GGUF

Hardware
1× NVIDIA GeForce RTX 5090
Workload
1024 → 256
Runs
8 eligible
Efficient frontierOther eligible run
p95 TTFT (ms)Request throughput (req/s)037309.802run_bWTq0O_nnAnxgxfd: 1.03 req/s, p95 TTFT 20,762.2 msrun_EN8MGeFqXDOyl0ZF: 1.18 req/s, p95 TTFT 5,500.6 msrun_yeMg6oYp5EfUiY2l: 0.63 req/s, p95 TTFT 1,968.5 msrun_P_GKD01Hc9vMRq35: 0.38 req/s, p95 TTFT 1,059.8 msrun_cUVlMx5L6vXLgyiV: 1.94 req/s, p95 TTFT 37,309.8 msrun_UZeOCz_OkZ28rxnH: 0.79 req/s, p95 TTFT 10,886.4 msrun_iZhLAUPL-5P3C9zu: 0.76 req/s, p95 TTFT 2,951.2 msrun_FR4A4S6PaeJtiExH: 0.25 req/s, p95 TTFT 501.9 ms

Runs

All published runs with the same model build, accelerator setup, and request shape. Benchmark methodology remains attached to each run.

RuntimeMaterial parametersTest loadReq/sOutput tok/sp95 TTFTp95 TPOTp95 E2EFailuresRun
llama.cpp b10353dflash=false kv_cache=Q8_0 slots_per_replica=128 context_tokens_per_slot=2048 context_tokens_per_replica=262144c=64264.520,762.2 ms234.22 ms78,612.3 ms0run_bWTq0O_nnAnxgxfd
llama.cpp b10353dflash=false kv_cache=Q8_0 slots_per_replica=128 context_tokens_per_slot=2048 context_tokens_per_replica=262144c=16302.75,500.6 ms44.72 ms15,760.9 ms0run_EN8MGeFqXDOyl0ZF
llama.cpp b10353dflash=false kv_cache=Q8_0 slots_per_replica=128 context_tokens_per_slot=2048 context_tokens_per_replica=262144c=4160.61,968.5 ms17.44 ms6,402.6 ms0run_yeMg6oYp5EfUiY2l
llama.cpp b10353dflash=false kv_cache=Q8_0 slots_per_replica=128 context_tokens_per_slot=2048 context_tokens_per_replica=262144c=296.91,059.8 ms29.22 ms8,450.4 ms0run_P_GKD01Hc9vMRq35
llama.cpp b10353dflash=false kv_cache=Q8_0 slots_per_replica=128 context_tokens_per_slot=2048 context_tokens_per_replica=262144c=12849737,309.8 ms238.74 ms98,247.6 ms0run_cUVlMx5L6vXLgyiV
llama.cpp b10353dflash=false kv_cache=Q8_0 slots_per_replica=128 context_tokens_per_slot=2048 context_tokens_per_replica=262144c=3220210,886.4 ms157.39 ms48,983.7 ms0run_UZeOCz_OkZ28rxnH
llama.cpp b10353dflash=false kv_cache=Q8_0 slots_per_replica=128 context_tokens_per_slot=2048 context_tokens_per_replica=262144c=8193.52,951.2 ms37.11 ms11,041.2 ms0run_iZhLAUPL-5P3C9zu
llama.cpp b10353dflash=false kv_cache=Q8_0 slots_per_replica=128 context_tokens_per_slot=2048 context_tokens_per_replica=262144c=164.9501.9 ms14.08 ms4,089.7 ms0run_FR4A4S6PaeJtiExH
Runtime
llama.cpp b10353
Material parameters
dflash=false kv_cache=Q8_0 slots_per_replica=128 context_tokens_per_slot=2048 context_tokens_per_replica=262144
Test load
c=64
Req/s
Output tok/s
264.5
p95 TTFT / TPOT
20,762.2 ms / 234.22 ms
Runtime
llama.cpp b10353
Material parameters
dflash=false kv_cache=Q8_0 slots_per_replica=128 context_tokens_per_slot=2048 context_tokens_per_replica=262144
Test load
c=16
Req/s
Output tok/s
302.7
p95 TTFT / TPOT
5,500.6 ms / 44.72 ms
Runtime
llama.cpp b10353
Material parameters
dflash=false kv_cache=Q8_0 slots_per_replica=128 context_tokens_per_slot=2048 context_tokens_per_replica=262144
Test load
c=4
Req/s
Output tok/s
160.6
p95 TTFT / TPOT
1,968.5 ms / 17.44 ms
Runtime
llama.cpp b10353
Material parameters
dflash=false kv_cache=Q8_0 slots_per_replica=128 context_tokens_per_slot=2048 context_tokens_per_replica=262144
Test load
c=2
Req/s
Output tok/s
96.9
p95 TTFT / TPOT
1,059.8 ms / 29.22 ms
Runtime
llama.cpp b10353
Material parameters
dflash=false kv_cache=Q8_0 slots_per_replica=128 context_tokens_per_slot=2048 context_tokens_per_replica=262144
Test load
c=128
Req/s
Output tok/s
497
p95 TTFT / TPOT
37,309.8 ms / 238.74 ms
Runtime
llama.cpp b10353
Material parameters
dflash=false kv_cache=Q8_0 slots_per_replica=128 context_tokens_per_slot=2048 context_tokens_per_replica=262144
Test load
c=32
Req/s
Output tok/s
202
p95 TTFT / TPOT
10,886.4 ms / 157.39 ms
Runtime
llama.cpp b10353
Material parameters
dflash=false kv_cache=Q8_0 slots_per_replica=128 context_tokens_per_slot=2048 context_tokens_per_replica=262144
Test load
c=8
Req/s
Output tok/s
193.5
p95 TTFT / TPOT
2,951.2 ms / 37.11 ms
Runtime
llama.cpp b10353
Material parameters
dflash=false kv_cache=Q8_0 slots_per_replica=128 context_tokens_per_slot=2048 context_tokens_per_replica=262144
Test load
c=1
Req/s
Output tok/s
64.9
p95 TTFT / TPOT
501.9 ms / 14.08 ms