ABOUT INFERX

    We rebuilt inference from the runtime up.

    InferX restores initialized CPU and GPU runtime state on demand, making true serverless inference possible without keeping GPUs running while idle.
    Sub-second cold starts for 32B (64GB) models.View benchmark
    Traditional startup
    1. Load weights
    2. Initialize framework
    3. Allocate GPU
    4. Serve
    InferX
    1. Restore initialized state
    2. Attach GPU
    3. Serve
    Sub-second cold starts for 32B (64GB) models.Runtime state restored on demand instead of rebuilding from disk.

    The systems problem

    Inference has been treated as a serving problem.

    Traditional inference platforms either keep GPUs running continuously or force users to wait through model loading and framework initialization.

    Always-on

    Fast response requires keeping capacity allocated.

    1. GPU reserved continuously
    2. Idle capacity remains billable
    3. Fast response requires over-provisioning

    Conventional scale-to-zero

    Capacity can stop, but the startup path rebuilds too much state.

    1. Download model
    2. Load weights
    3. Initialize framework
    4. Allocate GPU
    5. Long cold start

    InferX

    Initialized runtime state is restored on demand.

    1. Restore initialized CPU + GPU state
    2. Attach compute
    3. Begin inference
    4. Scale back to zero

    What InferX built

    A runtime designed for inference.

    InferX is not a wrapper or thin serving layer. It manages initialized model state and compute allocation as first-class runtime responsibilities.

    API request

    OpenAI-compatible request enters the control plane.

    InferX scheduler

    Selects the model instance and required resources.

    Initialized snapshot

    Restores initialized CPU and GPU runtime state.

    Secure runtime

    Runs inference inside isolated runtime boundaries.

    GPU allocation

    Attaches compute only while the workload is active.

    Token stream

    Streams tokens back through the API.

    Scale to zero

    Releases active GPU capacity between workloads.

    Technical proof

    Measured on production hardware.

    The public benchmark record separates measured observations from unpublished fields instead of filling gaps with inferred numbers.
    <1sGPU Snapshot Restore

    Published observation on NVIDIA H100

    Qwen3.6-35B-A3BModel

    FP16 · ~70 GB public benchmark configuration

    Scale to zeroRuntime lifecycle

    GPU released after active inference

    OpenAI-compatibleAPI surface

    Use standard client patterns

    Full benchmark methodology Explore architecture

    Economics

    Pay for inference, not idle infrastructure.

    The runtime shortens the active compute window: requests attach GPU capacity when work arrives, then release it between workloads.
    Traditional

    Allocated continuously across requests and idle periods.

    InferX

    GPU active only around inference requests, then released.

    • No idle GPU burn
    • Scale-to-zero economics
    • On-demand custom models
    • Faster access to initialized models

    The team

    Built by systems engineers.

    InferX is built by engineers with direct experience across distributed systems, cloud architecture, GPU infrastructure, runtime engineering, and production AI systems.
    BS

    Brad Sun

    ex-Chief Architect, Huawei Cloud

    Has spent the last 7+ years building the core runtime behind InferX from the ground up.

    JQ

    Jason Qu

    ex-Principal Architect, Microsoft

    Brings 15+ years of experience in distributed systems and runtime engineering.

    Meet the leadership team

    Infrastructure should disappear behind the API.

    Use production-ready endpoints, deploy custom models serverlessly, or run the InferX Platform in your own infrastructure.