BENCHMARKS

    Measured performance.
    Not marketing claims.

    Every benchmark includes hardware, model, methodology, and reproducible assumptions.

    Read methodology
    01

    Cold Start

    End-to-end startup measured from request admission to a model-ready runtime.

    TEST CONFIGURATION
    MODEL
    Qwen3.6-35B-A3B
    PRECISION
    FP16
    MODEL SIZE
    ~70 GB
    HARDWARE
    NVIDIA H100
    TRADITIONAL STARTUPLoad Model Weights≈35s
    INFERX STARTUPGPU Snapshot Restore<1s
    0s10s20s30s40s

    Measured on production hardware. Full environmental fields remain listed below; unpublished fields are not inferred.

    02

    Startup Breakdown

    The time difference comes from changing the work performed before the first request can run.

    TRADITIONALRebuild execution state
    1. 01Reserve GPU
    2. 02Load weights
    3. 03Initialize runtime
    4. 04Warm KV cache
    5. 05Ready
    INFERXRestore initialized state
    1. 01Scheduler
    2. 02Restore snapshot
    3. 03Attach GPU
    4. 04Ready
    03

    GPU Snapshot Restore

    Select a model class and accelerator to inspect published restore measurements.

    MODEL CLASS
    ACCELERATOR
    PUBLISHED OBSERVATION<1s

    GPU Snapshot Restore · ~70 GB FP16 model · NVIDIA H100

    04

    Scale to Zero

    Model readiness is retained while active GPU capacity is released between bursts.

    1. 01IdleNO ACTIVE GPU
    2. 02Snapshot retainedNO ACTIVE GPU
    3. 03GPU attachedGPU ALLOCATED
    4. 04InferenceGPU ALLOCATED
    5. 05GPU releasedNO ACTIVE GPU
    6. 06Snapshot retainedNO ACTIVE GPU
    05

    Throughput

    Published serving measurements include workload shape, request volume, failure behavior, and latency context.

    PUBLISHED REPORT · FIXED 100 RPM

    Xiaomi MiMo v2.5 TPM Ramp

    Target TPM increased from 0.10M to 30.00M by enlarging prompt tokens per request.

    Download source PDF
    HIGHEST ZERO-ERROR TARGET25.00Mtarget TPM
    HIGHEST OBSERVED SUCCESS18.41MTPM at 30.00M target
    OVERALL SUCCESS RATE99.67%1,495 / 1,500 requests
    HIGHEST P95 TTFT7.35sat 30.00M target TPM
    FIRST DEGRADED STAGE22.00M target TPM
    CACHE READ RATIO99.81%106,788,608 cache-read tokens
    FAILURE PROFILE5 failed requestsMainly non-2xx streaming responses; no timeout or network-error stage observed.
    INTERPRETATION NOTE

    The ramp was non-monotonic. The report treats this run as a rate-limit, window, and queueing observation—not a standalone absolute TPM ceiling.

    Concurrent requests
    AWAITING VERIFIED DATA
    GPU utilization
    AWAITING VERIFIED DATA
    06

    Multi-GPU Scaling

    Scaling efficiency requires matched model, interconnect, tensor-parallel, and request settings.

    GPU COUNTTOPOLOGYTHROUGHPUTSCALING EFFICIENCYSTATUS
    1Not published——Pending validated run
    2Not published——Pending validated run
    4Not published——Pending validated run
    8Not published——Pending validated run
    07

    Hardware Used

    The current public record contains the fields below. Missing fields are shown explicitly.

    FIELDVALUEDISCLOSURE STATUS
    HardwareNVIDIA H100Published
    DriverNot publishedRequired for reproduction
    CUDANot publishedRequired for reproduction
    vLLM versionNot publishedRequired for reproduction
    PrecisionFP16Published
    ModelQwen3.6-35B-A3BPublished
    PromptNot publishedRequired for reproduction
    Batch sizeNot publishedRequired for reproduction
    08

    Methodology

    A benchmark is useful only when its boundaries and omissions are visible.

    01

    Scope

    Measure startup separately from steady-state serving. Do not combine model initialization with token generation throughput.

    02

    Timer boundary

    Cold-start timing begins when a request enters the startup path and ends when the initialized model runtime is ready to serve.

    03

    Matched conditions

    Comparisons must use the same model, precision, accelerator class, and readiness definition.

    04

    Repeated runs

    Publish run count, warm-up policy, distribution, and outlier handling with every production dataset.

    05

    No inferred values

    Unavailable driver, CUDA, framework, prompt, batch, and serving measurements remain explicitly unpublished.

    06

    Result labeling

    Separate measured observations from illustrative scenarios and derived estimates.

    09

    Reproduce

    The public harness and complete run manifest will be linked here when the remaining environment fields are published.

    benchmark-manifest.yaml
    # Public benchmark harness
status: pending publication

# Required run manifest
model: Qwen3.6-35B-A3B
precision: FP16
hardware: NVIDIA H100