Qwen3.8-27B available nowUse endpoint
    BUILT BY THE TEAM THAT SOLVED SUB-SECOND COLD STARTS FOR LLMs

    Own your inference stack.

    Deploy InferX on your own GPUs, Kubernetes cluster, private cloud, or GPU cloud. Or start with hosted endpoints to try the runtime today.

    Reduce inference infrastructure costs by 30-70%* with fast cold starts, higher GPU utilization, isolated workloads, and OpenAI-compatible APIs.

    Explore PlatformTry Hosted Endpoints

    A request enters the OpenAI-compatible API, the scheduler restores initialized model state, GPU capacity attaches while tokens stream, and resources return to the pool when serving completes.

    INFERX RUNTIMEOPENAI-COMPATIBLE API
    GPU SNAPSHOT READY
    RESOURCE LAYERCAPACITY IDLE
    Snapshot PoolInitialized model state
    GPU PoolGPU slicing enabled0 SLICES
    Secure RuntimeIsolated container
    Scale to ZeroCapacity released

    Compatible with your favorite AI tools

    OpenAILangChainOpenCodeHermesOpenClawContinueLlamaIndexVercel AI SDKDify
    Compatibility references do not imply partnership or endorsement.

    THREE WAYS TO BUILD

    Choose how you run inference.

    Start with a hosted API, deploy your own model serverlessly, or run the complete InferX Platform in your infrastructure.
    READY-MODEL APIsRun on InferX Runtime

    InferX Endpoints

    Access production-ready open models through OpenAI-compatible APIs. InferX manages the infrastructure, scaling, and runtime.

    • Start with an API key
    • Pay per token
    • No infrastructure to manage
    Explore Endpoints
    YOUR INFRASTRUCTURE

    InferX Platform

    Deploy the complete InferX inference platform in your infrastructure.

    • Own your inference stack
    • Bring your own GPUs or cloud
    • End-to-end inference platform
    • Run private AI inference in your environment
    • Reduce inference costs by 30-70%.
    • Serverless GPU runtime for on-prem and private cloud
    • Keep models, data, and infrastructure under your control
    Explore Platform DeploymentContact Sales
    CUSTOM MODELS

    True Serverless Inference

    Deploy your own model on InferX and pay only for the GPU compute used while serving requests.

    • Scale to zero when idle
    • No idle GPU cost
    • Sub-second cold starts
    • OpenAI-compatible APIs
    Deploy a Custom Model

    SECURITY POSTURE

    Enterprise-ready infrastructure.

    View Security

    Zero data retention

    Prompts, outputs, and API payloads are not stored or used for training.

    Encrypted in transit

    Requests and responses are protected with modern transport encryption.

    SOC 2 logo
    SOC 2Type II in progress
    HIPAA logo
    HIPAAReadiness targeted Q3 2026
    GDPR logo
    GDPRReadiness targeted Q3 2026

    Production-ready endpoints.

    RUNNING ON INFERX ENDPOINTS RIGHT NOW
    Same runtime, same snapshot-restore scheduler — hosted for you to test before you deploy it yourself.

    Detailed input, output, and cached pricing lives on the Models page.

    View all models

    Our sub-sec cold starts, proven in production.

    MODEL
    Qwen3.6-35B-A3B
    PRECISION
    FP16
    MODEL SIZE
    ~70 GB
    HARDWARE
    NVIDIA H100
    TRADITIONAL STARTUPLoad model weights≈35s
    1. 01Pull container image9.5s
    2. 02Init runtime3s
    3. 03Load model weights17.5s
    4. 04Warm CUDA graphs3.5s
    5. 05Health check1.5s
    INFERX STARTUPGPU Snapshot Restore<1s
    1. 01Restore GPU snapshot<1s
    2. 02Rebind device memoryfast path
    3. 03Ready to serve0s idle

    SHORTER STARTUP PATH

    1. Request
    2. Snapshot restore
    3. Scale to zero

    Inside the InferX Runtime

    A model-ready runtime, restored on demand.

    Explore the complete architecture

    A request enters the OpenAI-compatible API, the scheduler restores initialized model state, GPU capacity attaches while tokens stream, and resources return to the pool when serving completes.

    INFERX RUNTIMEOPENAI-COMPATIBLE API
    GPU SNAPSHOT READY
    RESOURCE LAYERCAPACITY IDLE
    Snapshot PoolInitialized model state
    GPU PoolGPU slicing enabled0 SLICES
    Secure RuntimeIsolated container
    Scale to ZeroCapacity released

    OpenAI compatible

    Change one line. Keep your OpenAI code.

    Keep the OpenAI Python SDK and request shape. Point your existing client at InferX to serve open models in production.

    Verified SDK compatibility
    Get API KeyRead Docs
    inference.py
    PYTHON 3.12
    import osfrom openai import OpenAIclient = OpenAI(    api_key=os.environ["OPENAI_API_KEY"],    base_url="https://model.inferx.net/v1",)stream = client.chat.completions.create(    model="Qwen/Qwen3.6-35B-A3B",    messages=[{"role": "user", "content": "Hello, InferX."}],    stream=True,)for chunk in stream:    print(chunk.choices[0].delta.content or "", end="")
    InferX multi-tenant secure container runtime diagram showing guest user, guest kernel, host user, host kernel, and NCCL security isolation layers.

    Put production inference on a runtime built for it.

    Start BuildingContact Sales