Point us at your GPUs
Kubernetes, bare metal, private cloud, or your own cloud account.
BUILT BY THE TEAM THAT SOLVED SUB-SECOND COLD STARTS FOR LLMs
Own your inference stack.
Deploy InferX on your own GPUs, Kubernetes cluster, private cloud, or GPU cloud. Or start with hosted endpoints to try the runtime today.
Reduce inference infrastructure costs by 30-70%* with fast cold starts, higher GPU utilization, isolated workloads, and OpenAI-compatible APIs.
A request enters the OpenAI-compatible API, the scheduler restores initialized model state, GPU capacity attaches while tokens stream, and resources return to the pool when serving completes.
INFERX RUNTIMEOPENAI-COMPATIBLE API
GPU SNAPSHOT READYRESOURCE LAYERCAPACITY IDLE
Snapshot PoolInitialized model state
GPU PoolGPU slicing enabled0 SLICES
Secure RuntimeIsolated container
Scale to ZeroCapacity released
Compatible with your favorite AI tools
OpenAILangChainOpenCodeHermesOpenClawContinueLlamaIndexVercel AI SDKDify
Build vs. Deploy
Deploy this week. Not this year.
What it takes
Build in-house
InferX
Team
Dedicated runtime engineering team
Zero engineering headcount
Time to production
Years
Days to weeks
Scale
Re-engineered at every stage of growth
1 GPU to 1,000s, same runtime
Cost
Team salaries + idle GPU spend, indefinitely
30–70%* lower infrastructure cost, zero engineering overhead
What you're solving
A problem already solved
Already solved, already running
Ongoing
Permanent maintenance
We maintain it. You ship product.
Building this is a solved problem.The only question is how fast you want it running.*Savings depend on workload patterns, model size, traffic shape, and existing GPU utilization.
The simple path
Plug in. Not build out.
No migration. No new infrastructure to stand up. No engineering sprint.
We install the runtime
No changes to your existing stack or workloads.
You're serving in production
Sub-second cold starts, isolated workloads, live in days.
Zero data retention
Prompts, outputs, and API payloads are not stored or used for training.
Encrypted in transit
Requests and responses are protected with modern transport encryption.
SOC 2Type II in progress
HIPAAReadiness targeted Q3 2026
GDPRReadiness targeted Q3 2026
Our sub-sec cold starts, proven in production.
- MODEL
- Qwen3.6-35B-A3B
- PRECISION
- FP16
- MODEL SIZE
- ~70 GB
- HARDWARE
- NVIDIA H100
TRADITIONAL STARTUPLoad model weights≈35s
- 01Pull container image9.5s
- 02Init runtime3s
- 03Load model weights17.5s
- 04Warm CUDA graphs3.5s
- 05Health check1.5s
INFERX STARTUPGPU Snapshot Restore<1s
- 01Restore GPU snapshot<1s
- 02Rebind device memoryfast path
- 03Ready to serve0s idle
SHORTER STARTUP PATH
- Request
- Snapshot restore
- Scale to zero
Inside the InferX Runtime
A model-ready runtime, restored on demand.
A request enters the OpenAI-compatible API, the scheduler restores initialized model state, GPU capacity attaches while tokens stream, and resources return to the pool when serving completes.
INFERX RUNTIMEOPENAI-COMPATIBLE API
GPU SNAPSHOT READYRESOURCE LAYERCAPACITY IDLE
Snapshot PoolInitialized model state
GPU PoolGPU slicing enabled0 SLICES
Secure RuntimeIsolated container
Scale to ZeroCapacity released
Production-ready endpoints.
RUNNING ON INFERX ENDPOINTS RIGHT NOW
Same runtime, same snapshot-restore scheduler — hosted for you to test before you deploy it yourself.
GLM 5.3 Flash
Qwen 3.8 27B
DeepSeek V4 Flash 0731
DeepSeek V4.1 Flash
Qwen3.6 27B FP8
Qwen3.6 35B A3B FP8
Gemma 4 31B IT FP8
GPT-OSS 20B
OpenAI compatible
Change one line. Keep your OpenAI code.
Keep the OpenAI Python SDK and request shape. Point your existing client at InferX to serve open models in production.
Verified SDK compatibilityinference.py
PYTHON 3.12
import osfrom openai import OpenAIclient = OpenAI( api_key=os.environ["OPENAI_API_KEY"], base_url="https://model.inferx.net/v1",)stream = client.chat.completions.create( model="Qwen/Qwen3.6-35B-A3B", messages=[{"role": "user", "content": "Hello, InferX."}], stream=True,)for chunk in stream: print(chunk.choices[0].delta.content or "", end="")