Point us at your GPUs
Kubernetes, bare metal, private cloud, or your own cloud account.
BUILT BY THE TEAM THAT SOLVED SUB-SECOND COLD STARTS FOR LLMs
Own your inference stack.
Deploy InferX Platform on your own GPUs, Kubernetes cluster, private cloud, or GPU cloud. Or access InferX Public Endpoints to try hosted models without managing GPU infrastructure.
Reduce inference infrastructure costs by 30-70%* with fast cold starts, higher GPU utilization, isolated workloads, and OpenAI-compatible APIs.
A request enters the OpenAI-compatible API, the scheduler restores initialized model state, GPU capacity attaches while tokens stream, and resources return to the pool when serving completes.
INFERX RUNTIMEOPENAI-COMPATIBLE API
GPU SNAPSHOT READYRESOURCE LAYERCAPACITY IDLE
Snapshot PoolInitialized model state
GPU PoolGPU slicing enabled0 SLICES
Secure RuntimeIsolated container
Scale to ZeroCapacity released
Compatible with your favorite AI tools
OpenAILangChainOpenCodeHermesOpenClawContinueLlamaIndexVercel AI SDKDify
Build vs. Deploy
Deploy this week. Not this year.
What it takes
Build in-house
InferX
Team
Dedicated runtime engineering team
Zero engineering headcount
Time to production
Years
Days to weeks
Scale
Re-engineered at every stage of growth
1 GPU to 1,000s, flexible deployment
Cost
Team salaries + idle GPU spend, indefinitely
30–70%* lower infrastructure cost, zero engineering overhead
What you're solving
A problem already solved
Already solved, already running
Ongoing
Permanent maintenance
We maintain it. You ship product.
Building this is a solved problem.The only question is how fast you want it running.*Savings depend on workload patterns, model size, traffic shape, and existing GPU utilization.
The simple path
Plug in. Not build out.
No migration. No new infrastructure to stand up. No engineering sprint.
We install the runtime
No changes to your existing stack or workloads.
You're serving in production
Sub-second cold starts, isolated workloads, live in days.
SECURITY POSTURE
Enterprise Security for InferX Platform
Built for enterprise inference deployments, InferX Platform provides workload isolation, secure execution, and infrastructure-level controls designed to protect customer workloads.
View SecurityCustomer-Controlled Data
Deploy InferX Platform within your infrastructure and maintain control over inference data, access policies, and retention settings.
Encrypted in transit
Requests and responses are protected with modern transport encryption.
SOC 2Type II in progress
HIPAAReadiness targeted Q4 2026
GDPRReadiness targeted Q4 2026
Our sub-sec cold starts, proven in production.
- MODEL
- Qwen3.6-35B-A3B
- PRECISION
- FP16
- MODEL SIZE
- ~70 GB
- HARDWARE
- NVIDIA H100
TRADITIONAL STARTUPLoad model weights≈35s
- 01Pull container image9.5s
- 02Init runtime3s
- 03Load model weights17.5s
- 04Warm CUDA graphs3.5s
- 05Health check1.5s
INFERX STARTUPGPU Snapshot Restore<1s
- 01Restore GPU snapshot<1s
- 02Rebind device memoryfast path
- 03Ready to serve0s idle
SHORTER STARTUP PATH
- Request
- Snapshot restore
- Scale to zero
Inside the InferX Runtime
A model-ready runtime, restored on demand.
A request enters the OpenAI-compatible API, the scheduler restores initialized model state, GPU capacity attaches while tokens stream, and resources return to the pool when serving completes.
INFERX RUNTIMEOPENAI-COMPATIBLE API
GPU SNAPSHOT READYRESOURCE LAYERCAPACITY IDLE
Snapshot PoolInitialized model state
GPU PoolGPU slicing enabled0 SLICES
Secure RuntimeIsolated container
Scale to ZeroCapacity released
Production-ready endpoints.
RUNNING ON INFERX ENDPOINTS RIGHT NOW
Explore hosted model APIs through InferX Public Endpoints, or deploy InferX Platform on your own infrastructure.
GLM 5.3 Flash
GLM-5.3
Kimi K3
Qwen3.8 Max
Qwen 3.8 27B
DeepSeek V4 Flash 0731
DeepSeek V4.1 Flash 1017
Qwen3.6 27B FP8
Qwen3.6 35B A3B FP8
Gemma 4 31B IT FP8
GPT-OSS 20B
OpenAI compatible
Change one line. Keep your OpenAI code.
Keep the OpenAI Python SDK and request shape. Point your existing client at InferX to serve open models in production.
Verified SDK compatibilityinference.py
PYTHON 3.12
import osfrom openai import OpenAIclient = OpenAI( api_key=os.environ["OPENAI_API_KEY"], base_url="https://model.inferx.net/v1",)stream = client.chat.completions.create( model="Qwen/Qwen3.6-35B-A3B", messages=[{"role": "user", "content": "Hello, InferX."}], stream=True,)for chunk in stream: print(chunk.choices[0].delta.content or "", end="")