funcpod
| tenant | namespace | podname |
|---|---|---|
| public | deepseek-ai | public/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B/234/1223 |
state
| state | time |
|---|---|
| Init | 2025/04/27 02:23:51 |
| Creating | 2025/04/27 02:23:51 |
| Standby | 2025/04/27 02:23:54 |
| Standby | 2025/04/27 02:23:54 |
| Resuming | 2025/04/27 19:09:04 |
| Ready | 2025/04/27 19:09:04 |
log
| INFO 04-27 12:09:05 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO 04-27 12:09:05 logger.py:36] Received request cmpl-f7c932388f544a19ac581dee07a257c3-0: prompt: 'what is the integral of x^2 from 0 to 2?\nPlease reason step by step, and put your final answer within \\boxed{}.', params: SamplingParams(n=1, best_of=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.0, top_p=1.0, top_k=-1, min_p=0.0, seed=None, use_beam_search=False, length_penalty=1.0, early_stopping=False, stop=[], stop_token_ids=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=1000, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None), prompt_token_ids: [151646, 12555, 374, 279, 25098, 315, 856, 61, 17, 504, 220, 15, 311, 220, 17, 5267, 5501, 2874, 3019, 553, 3019, 11, 323, 2182, 697, 1590, 4226, 2878, 1124, 79075, 46391], lora_request: None, prompt_adapter_request: None. INFO: 10.1.2.3:123 - "POST /v1/completions HTTP/1.1" 200 OK INFO 04-27 12:09:05 engine.py:288] Added request cmpl-f7c932388f544a19ac581dee07a257c3-0. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:09:10 metrics.py:351] Avg prompt throughput: 6.2 tokens/s, Avg generation throughput: 38.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:09:15 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 44.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:09:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 44.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:09:25 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 44.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:09:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 10.5 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:09:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:09:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:10:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:10:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:10:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:10:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:10:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:10:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:11:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:11:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:11:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:11:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:11:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:11:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:12:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:12:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:12:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:12:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:12:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:12:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:13:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:13:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:13:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:13:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:13:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:13:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:14:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:14:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:14:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:14:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:14:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:14:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:15:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:15:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:15:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:15:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:15:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:15:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:16:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:16:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:16:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:16:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:16:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:16:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:17:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:17:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:17:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:17:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:17:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:17:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:18:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:18:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:18:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:18:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:18:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:18:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:19:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:19:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:19:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:19:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:19:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:19:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:20:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:20:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:20:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:20:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:20:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:20:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:21:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:21:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:21:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:21:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:21:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:21:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:22:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:22:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:22:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:22:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:22:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:22:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:23:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:23:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:23:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:23:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:23:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:23:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:24:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:24:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:24:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:24:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:24:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:24:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:25:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:25:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:25:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:25:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:25:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:25:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:26:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:26:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:26:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:26:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:26:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:26:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:27:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:27:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:27:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:27:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:27:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:27:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:28:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:28:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:28:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:28:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:28:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:28:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:29:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:29:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:29:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:29:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:29:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:29:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:30:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:30:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:30:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:30:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:30:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:30:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:31:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:31:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:31:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:31:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:31:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:31:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:32:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:32:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:32:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:32:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:32:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:32:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:33:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:33:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:33:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:33:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:33:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:33:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:34:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:34:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:34:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:34:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:34:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:34:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:35:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:35:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:35:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:35:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:35:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:35:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:36:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:36:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:36:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:36:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:36:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:36:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:37:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:37:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:37:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:37:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:37:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:37:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:38:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:38:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:38:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:38:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:38:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:38:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:39:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:39:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:39:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:39:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:39:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:39:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:40:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:40:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:40:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:40:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:40:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:40:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:41:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:41:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:41:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:41:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:41:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:41:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:42:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:42:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:42:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:42:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:42:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:42:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:43:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:43:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:43:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:43:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:43:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:43:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:44:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:44:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:44:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:44:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:44:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:44:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:45:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:45:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:45:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:45:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:45:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:45:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:46:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:46:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:46:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:46:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:46:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:46:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:47:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:47:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:47:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:47:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:47:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:47:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:48:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:48:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:48:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:48:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:48:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:48:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:49:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:49:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:49:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:49:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:49:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:49:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:50:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:50:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:50:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:50:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:50:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:50:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:51:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:51:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:51:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:51:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:51:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:51:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:52:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:52:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:52:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:52:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:52:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:52:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:53:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:53:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:53:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:53:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:53:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:53:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:54:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:54:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:54:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:54:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:54:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:54:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:55:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:55:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:55:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:55:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:55:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:55:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:56:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:56:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:56:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:56:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:56:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:56:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:57:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:57:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:57:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:57:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:57:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:57:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:58:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:58:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:58:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:58:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:58:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:58:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:59:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:59:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:59:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:59:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:59:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 12:59:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:00:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:00:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:00:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:00:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:00:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:00:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:01:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:01:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:01:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:01:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:01:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:01:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:02:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:02:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:02:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:02:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:02:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:02:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:03:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:03:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:03:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:03:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:03:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:03:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:04:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:04:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:04:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:04:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:04:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:04:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:05:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:05:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:05:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:05:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:05:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:05:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:06:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:06:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:06:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:06:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:06:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:06:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:07:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:07:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:07:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:07:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:07:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:07:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:08:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:08:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:08:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:08:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:08:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:08:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:09:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:09:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:09:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:09:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:09:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:09:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:10:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:10:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:10:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:10:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:10:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:10:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:11:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:11:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:11:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:11:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:11:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:11:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:12:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:12:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:12:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:12:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:12:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:12:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:13:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:13:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:13:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:13:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:13:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:13:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:14:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:14:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:14:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:14:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:14:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:14:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:15:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:15:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:15:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:15:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:15:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:15:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:16:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:16:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:16:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:16:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:16:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:16:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:17:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:17:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:17:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:17:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:17:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:17:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:18:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:18:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:18:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:18:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:18:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:18:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:19:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:19:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:19:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:19:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:19:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:19:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:20:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:20:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:20:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:20:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:20:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:20:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:21:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:21:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:21:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:21:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:21:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:21:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:22:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:22:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:22:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:22:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:22:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:22:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:23:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:23:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:23:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:23:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:23:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:23:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:24:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:24:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:24:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:24:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:24:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:24:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:25:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:25:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:25:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:25:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:25:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:25:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:26:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:26:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:26:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:26:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:26:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:26:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:27:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:27:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:27:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:27:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:27:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:27:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:28:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:28:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:28:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:28:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:28:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:28:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:29:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:29:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:29:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:29:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:29:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:29:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:30:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:30:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:30:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:30:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:30:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:30:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:31:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:31:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:31:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:31:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:31:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:31:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:32:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:32:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:32:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:32:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:32:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:32:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:33:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:33:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:33:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:33:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:33:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:33:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:34:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:34:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:34:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:34:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:34:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:34:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:35:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:35:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:35:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:35:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:35:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:35:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:36:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:36:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:36:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:36:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:36:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:36:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:37:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:37:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:37:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:37:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:37:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:37:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:38:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:38:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:38:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:38:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:38:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:38:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:39:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:39:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:39:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:39:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:39:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:39:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:40:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:40:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:40:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:40:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:40:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:40:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:41:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:41:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:41:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:41:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:41:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:41:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:42:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:42:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:42:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:42:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:42:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:42:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:43:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:43:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:43:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:43:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:43:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:43:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:44:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:44:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:44:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:44:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:44:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:44:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:45:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:45:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:45:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:45:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:45:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:45:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:46:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:46:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:46:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:46:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:46:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:46:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:47:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:47:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:47:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:47:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:47:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:47:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:48:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:48:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:48:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:48:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:48:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:48:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:49:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:49:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:49:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:49:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:49:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:49:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:50:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:50:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:50:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:50:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:50:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:50:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:51:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:51:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:51:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:51:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:51:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:51:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:52:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:52:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:52:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:52:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:52:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:52:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:53:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:53:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:53:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:53:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:53:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:53:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:54:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:54:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:54:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:54:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:54:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:54:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:55:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:55:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:55:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:55:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:55:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:55:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:56:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:56:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:56:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:56:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:56:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:56:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:57:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:57:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:57:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:57:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:57:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:57:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:58:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:58:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:58:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:58:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:58:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:58:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:59:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:59:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:59:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:59:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:59:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 13:59:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:00:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:00:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:00:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:00:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:00:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:00:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:01:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:01:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:01:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:01:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:01:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:01:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:02:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:02:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:02:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:02:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:02:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:02:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:03:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:03:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:03:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:03:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:03:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:03:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:04:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:04:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:04:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:04:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:04:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:04:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:05:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:05:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:05:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:05:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:05:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:05:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:06:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:06:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:06:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:06:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:06:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:06:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:07:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:07:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:07:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:07:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:07:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:07:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:08:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:08:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:08:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:08:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:08:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:08:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:09:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:09:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:09:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:09:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:09:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:09:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:10:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:10:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:10:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:10:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:10:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:10:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:11:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:11:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:11:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:11:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:11:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:11:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:12:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:12:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:12:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:12:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:12:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:12:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:13:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:13:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:13:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:13:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:13:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:13:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:14:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:14:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:14:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:14:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:14:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:14:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:15:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:15:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:15:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:15:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:15:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:15:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:16:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:16:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:16:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:16:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:16:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:16:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:17:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:17:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:17:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:17:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:17:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:17:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:18:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:18:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:18:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:18:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:18:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:18:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:19:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:19:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:19:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:19:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:19:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:19:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:20:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:20:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:20:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:20:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:20:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:20:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:21:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:21:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:21:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:21:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:21:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:21:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:22:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:22:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:22:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:22:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:22:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:22:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:23:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:23:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:23:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:23:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:23:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:23:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:24:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:24:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:24:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:24:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:24:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:24:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:25:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:25:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:25:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:25:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:25:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:25:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:26:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:26:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:26:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:26:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:26:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:26:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:27:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:27:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:27:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:27:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:27:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:27:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:28:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:28:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:28:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:28:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:28:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:28:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:29:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:29:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:29:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:29:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:29:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:29:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:30:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:30:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:30:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:30:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:30:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:30:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:31:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:31:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:31:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:31:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:31:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:31:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:32:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:32:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:32:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:32:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:32:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:32:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:33:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:33:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:33:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:33:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:33:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:33:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:34:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:34:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:34:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:34:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:34:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:34:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:35:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:35:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:35:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:35:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:35:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:35:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:36:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:36:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:36:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:36:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:36:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:36:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:37:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:37:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:37:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:37:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:37:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:37:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:38:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:38:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:38:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:38:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:38:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:38:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:39:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:39:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:39:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:39:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:39:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:39:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:40:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:40:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:40:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:40:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:40:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:40:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:41:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:41:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:41:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:41:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:41:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:41:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:42:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:42:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:42:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:42:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:42:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:42:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:43:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:43:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:43:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:43:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:43:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:43:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:44:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:44:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:44:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:44:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:44:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:44:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:45:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:45:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:45:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:45:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:45:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:45:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:46:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:46:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:46:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:46:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:46:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:46:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:47:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:47:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:47:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:47:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:47:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:47:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:48:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:48:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:48:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:48:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:48:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:48:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:49:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:49:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:49:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:49:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:49:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:49:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:50:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:50:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:50:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:50:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:50:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:50:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:51:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:51:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:51:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:51:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:51:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:51:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:52:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:52:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:52:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:52:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:52:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:52:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:53:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:53:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:53:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:53:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:53:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:53:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:54:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:54:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:54:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:54:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:54:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:54:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:55:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:55:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:55:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:55:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:55:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:55:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:56:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:56:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:56:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:56:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:56:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:56:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:57:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:57:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:57:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:57:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:57:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:57:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:58:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:58:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:58:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:58:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:58:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:58:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:59:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:59:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:59:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:59:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:59:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 14:59:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:00:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:00:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:00:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:00:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:00:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:00:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:01:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:01:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:01:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:01:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:01:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:01:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:02:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:02:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:02:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:02:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:02:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:02:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:03:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:03:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:03:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:03:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:03:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:03:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:04:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:04:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:04:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:04:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:04:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:04:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:05:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:05:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:05:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:05:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:05:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:05:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:06:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:06:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:06:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:06:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:06:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:06:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:07:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:07:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:07:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:07:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:07:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:07:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:08:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:08:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:08:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:08:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:08:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:08:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:09:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:09:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:09:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:09:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:09:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:09:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:10:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:10:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:10:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:10:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:10:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:10:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:11:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:11:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:11:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:11:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:11:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:11:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:12:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:12:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:12:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:12:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:12:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:12:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:13:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:13:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:13:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:13:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:13:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:13:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:14:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:14:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:14:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:14:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:14:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:14:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:15:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:15:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:15:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:15:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:15:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:15:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:16:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:16:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:16:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:16:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:16:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:16:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:17:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:17:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:17:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:17:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:17:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:17:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:18:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:18:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:18:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:18:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:18:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:18:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:19:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:19:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:19:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:19:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:19:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:19:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:20:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:20:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:20:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:20:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:20:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:20:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:21:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:21:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:21:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:21:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:21:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:21:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:22:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:22:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:22:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:22:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:22:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:22:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:23:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:23:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:23:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:23:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:23:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:23:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:24:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:24:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:24:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:24:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:24:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:24:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:25:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:25:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:25:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:25:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:25:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:25:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:26:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:26:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:26:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:26:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:26:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:26:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:27:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:27:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:27:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:27:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:27:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:27:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:28:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:28:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:28:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:28:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:28:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:28:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:29:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:29:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:29:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:29:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:29:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:29:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:30:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:30:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:30:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:30:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:30:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:30:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:31:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:31:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:31:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:31:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:31:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:31:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:32:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:32:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:32:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:32:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:32:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:32:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:33:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:33:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:33:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:33:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:33:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:33:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:34:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:34:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:34:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:34:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:34:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:34:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:35:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:35:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:35:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:35:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:35:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:35:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:36:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:36:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:36:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:36:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:36:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:36:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:37:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:37:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:37:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:37:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:37:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:37:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:38:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:38:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:38:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:38:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:38:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:38:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:39:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:39:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:39:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:39:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:39:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:39:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:40:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:40:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:40:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:40:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:40:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:40:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:41:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:41:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:41:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:41:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:41:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:41:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:42:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:42:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:42:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:42:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:42:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:42:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:43:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:43:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:43:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:43:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:43:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:43:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:44:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:44:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:44:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:44:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:44:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:44:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:45:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:45:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:45:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:45:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:45:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:45:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:46:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:46:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:46:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:46:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:46:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:46:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:47:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:47:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:47:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:47:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:47:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:47:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:48:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:48:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:48:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:48:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:48:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:48:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:49:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:49:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:49:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:49:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:49:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:49:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:50:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:50:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:50:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:50:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:50:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:50:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:51:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:51:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:51:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:51:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:51:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:51:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:52:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:52:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:52:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:52:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:52:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:52:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:53:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:53:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:53:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:53:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:53:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:53:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:54:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:54:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:54:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:54:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:54:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:54:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:55:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:55:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:55:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:55:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:55:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:55:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:56:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:56:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:56:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:56:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:56:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:56:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:57:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:57:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:57:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:57:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:57:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:57:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:58:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:58:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:58:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:58:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:58:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:58:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:59:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:59:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:59:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:59:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:59:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 15:59:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:00:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:00:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:00:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:00:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:00:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:00:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:01:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:01:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:01:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:01:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:01:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:01:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:02:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:02:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:02:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:02:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:02:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:02:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:03:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:03:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:03:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:03:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:03:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:03:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:04:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:04:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:04:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:04:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:04:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:04:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:05:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:05:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:05:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:05:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:05:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:05:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:06:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:06:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:06:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:06:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:06:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:06:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:07:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:07:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:07:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:07:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:07:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:07:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:08:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:08:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:08:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:08:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:08:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:08:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:09:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:09:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:09:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:09:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:09:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:10:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:10:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:10:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:10:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:10:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:10:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:11:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:11:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:11:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:11:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:11:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:11:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:12:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:12:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:12:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:12:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:12:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:12:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:13:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:13:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:13:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:13:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:13:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:13:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:14:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:14:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:14:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:14:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:14:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:14:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:15:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:15:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:15:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:15:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:15:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:15:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:16:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:16:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:16:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:16:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:16:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:16:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:17:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:17:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:17:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:17:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:17:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:17:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:18:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:18:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:18:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:18:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:18:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:18:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:19:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:19:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:19:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:19:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:19:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:19:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:20:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:20:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:20:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:20:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:20:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:20:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:21:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:21:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:21:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:21:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:21:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:21:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:22:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:22:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:22:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:22:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:22:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:22:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:23:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:23:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:23:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:23:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:23:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:23:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:24:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:24:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:24:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:24:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:24:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:24:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:25:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:25:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:25:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:25:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:25:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:25:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:26:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:26:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:26:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:26:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:26:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:26:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:27:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:27:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:27:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:27:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:27:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:27:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:28:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:28:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:28:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:28:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:28:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:28:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:29:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:29:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:29:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:29:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:29:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:29:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:30:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:30:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:30:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:30:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:30:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:30:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:31:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:31:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:31:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:31:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:31:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:31:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:32:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:32:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:32:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:32:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:32:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:32:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:33:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:33:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:33:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:33:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:33:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:33:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:34:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:34:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:34:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:34:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:34:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:34:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:35:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:35:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:35:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:35:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:35:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:35:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:36:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:36:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:36:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:36:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:36:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:36:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:37:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:37:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:37:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:37:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:37:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:37:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:38:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:38:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:38:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:38:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:38:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:38:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:39:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:39:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:39:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:39:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:39:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:39:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:40:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:40:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:40:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:40:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:40:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:40:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:41:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:41:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:41:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:41:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:41:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:41:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:42:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:42:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:42:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:42:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:42:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:42:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:43:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:43:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:43:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:43:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:43:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:43:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:44:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:44:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:44:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:44:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:44:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:44:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:45:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:45:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:45:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:45:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:45:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:45:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:46:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:46:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:46:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:46:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:46:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:46:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:47:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:47:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:47:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:47:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:47:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:47:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:48:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:48:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:48:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:48:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:48:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:48:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:49:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:49:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:49:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:49:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:49:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:49:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:50:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:50:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:50:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:50:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:50:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:50:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:51:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:51:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:51:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:51:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:51:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:51:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:52:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:52:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:52:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:52:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:52:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:52:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:53:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:53:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:53:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:53:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:53:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:53:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:54:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:54:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:54:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:54:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:54:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:54:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:55:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:55:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:55:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:55:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:55:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:55:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:56:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:56:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:56:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:56:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:56:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:56:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:57:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:57:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:57:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:57:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:57:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:57:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:58:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:58:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:58:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:58:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:58:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:58:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:59:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:59:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:59:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:59:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:59:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 16:59:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:00:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:00:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:00:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:00:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:00:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:00:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:01:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:01:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:01:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:01:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:01:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:01:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:02:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:02:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:02:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:02:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:02:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:02:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:03:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:03:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:03:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:03:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:03:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:03:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:04:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:04:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:04:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:04:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:04:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:04:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:05:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:05:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:05:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:05:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:05:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:05:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:06:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:06:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:06:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:06:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:06:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:06:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:07:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:07:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:07:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:07:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:07:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:07:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:08:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:08:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:08:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:08:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:08:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:08:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:09:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:09:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:09:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:09:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:09:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:09:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:10:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:10:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:10:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:10:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:10:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:10:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:11:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:11:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:11:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:11:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:11:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:11:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:12:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:12:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:12:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:12:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:12:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:12:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:13:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:13:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:13:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:13:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:13:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:13:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:14:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:14:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:14:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:14:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:14:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:14:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:15:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:15:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:15:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:15:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:15:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:15:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:16:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:16:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:16:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:16:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:16:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:16:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:17:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:17:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:17:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:17:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:17:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:17:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:18:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:18:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:18:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:18:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:18:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:18:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:19:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:19:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:19:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:19:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:19:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:19:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:20:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:20:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:20:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:20:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:20:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:20:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:21:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:21:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:21:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:21:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:21:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:21:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:22:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:22:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:22:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:22:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:22:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:22:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:23:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:23:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:23:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:23:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:23:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:23:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:24:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:24:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:24:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:24:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:24:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:24:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:25:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:25:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:25:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:25:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:25:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:25:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:26:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:26:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:26:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:26:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:26:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:26:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:27:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:27:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:27:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:27:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:27:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:27:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:28:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:28:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:28:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:28:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:28:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:28:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:29:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:29:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:29:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:29:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:29:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:29:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:30:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:30:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:30:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:30:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:30:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:30:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:31:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:31:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:31:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:31:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:31:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:31:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:32:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:32:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:32:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:32:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:32:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:32:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:33:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:33:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:33:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:33:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:33:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:33:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:34:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:34:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:34:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:34:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:34:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:34:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:35:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:35:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:35:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:35:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:35:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:35:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:36:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:36:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:36:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:36:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:36:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:36:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:37:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:37:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:37:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:37:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:37:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:37:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:38:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:38:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:38:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:38:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:38:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:38:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:39:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:39:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:39:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:39:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:39:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:39:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:40:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:40:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:40:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:40:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:40:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:40:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:41:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:41:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:41:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:41:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:41:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:41:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:42:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:42:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:42:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:42:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:42:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:42:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:43:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:43:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:43:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:43:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:43:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:43:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:44:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:44:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:44:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:44:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:44:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:44:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:45:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:45:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:45:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:45:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:45:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:45:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:46:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:46:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:46:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:46:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:46:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:46:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:47:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:47:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:47:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:47:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:47:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:47:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:48:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:48:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:48:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:48:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:48:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:48:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:49:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:49:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:49:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:49:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:49:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:49:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:50:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:50:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:50:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:50:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:50:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:50:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:51:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:51:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:51:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:51:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:51:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:51:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:52:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:52:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:52:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:52:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:52:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:52:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:53:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:53:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:53:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:53:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:53:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:53:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:54:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:54:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:54:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:54:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:54:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:54:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:55:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:55:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:55:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:55:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:55:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:55:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:56:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:56:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:56:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:56:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:56:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:56:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:57:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:57:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:57:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:57:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:57:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:57:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:58:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:58:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:58:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:58:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:58:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:58:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:59:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:59:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:59:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:59:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:59:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 17:59:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:00:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:00:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:00:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:00:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:00:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:00:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:01:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:01:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:01:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:01:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:01:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:01:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:02:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:02:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:02:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:02:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:02:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:02:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:03:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:03:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:03:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:03:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:03:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:03:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:04:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:04:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:04:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:04:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:04:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:04:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:05:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:05:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:05:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:05:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:05:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:05:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:06:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:06:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:06:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:06:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:06:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:06:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:07:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:07:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:07:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:07:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:07:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:07:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:08:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:08:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:08:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:08:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:08:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:08:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:09:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:09:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:09:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:09:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:09:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:09:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:10:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:10:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:10:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:10:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:10:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:10:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:11:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:11:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:11:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:11:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:11:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:11:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:12:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:12:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:12:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:12:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:12:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:12:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:13:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:13:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:13:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:13:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:13:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:13:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:14:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:14:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:14:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:14:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:14:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:14:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:15:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:15:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:15:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:15:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:15:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:15:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:16:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:16:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:16:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:16:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:16:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:16:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:17:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:17:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:17:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:17:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:17:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:17:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:18:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:18:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:18:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:18:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:18:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:18:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:19:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:19:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:19:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:19:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:19:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:19:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:20:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:20:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:20:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:20:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:20:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:20:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:21:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:21:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:21:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:21:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:21:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:21:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:22:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:22:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:22:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:22:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:22:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:22:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:23:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:23:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:23:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:23:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:23:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:23:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:24:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:24:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:24:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:24:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:24:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:24:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:25:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:25:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:25:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:25:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:25:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:25:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:26:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:26:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:26:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:26:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:26:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:26:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:27:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:27:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:27:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:27:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:27:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:27:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:28:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:28:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:28:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:28:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:28:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:28:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:29:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:29:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:29:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:29:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:29:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:29:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:30:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:30:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:30:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:30:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:30:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:30:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:31:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:31:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:31:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:31:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:31:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:31:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:32:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:32:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:32:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:32:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:32:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:32:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:33:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:33:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:33:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:33:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:33:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:33:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:34:00 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:34:10 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:34:20 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:34:30 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:34:40 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:34:50 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:35:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:35:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:35:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:35:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:35:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:35:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:36:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:36:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:36:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:36:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:36:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:36:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:37:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:37:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:37:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:37:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:37:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:37:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:38:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:38:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:38:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:38:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:38:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:38:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:39:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:39:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:39:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:39:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:39:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:39:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:40:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:40:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:40:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:40:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:40:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:40:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:41:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:41:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:41:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:41:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:41:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:41:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:42:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:42:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:42:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:42:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:42:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:42:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:43:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:43:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:43:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:43:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:43:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:43:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:44:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:44:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:44:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:44:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:44:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:44:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:45:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:45:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:45:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:45:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:45:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:45:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:46:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:46:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:46:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:46:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:46:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:46:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:47:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:47:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:47:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:47:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:47:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:47:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:48:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:48:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:48:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:48:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:48:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:48:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:49:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:49:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:49:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:49:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:49:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:49:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:50:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:50:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:50:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:50:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:50:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:50:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:51:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:51:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:51:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:51:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:51:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:51:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:52:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:52:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:52:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:52:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:52:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:52:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:53:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:53:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:53:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:53:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:53:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:53:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:54:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:54:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:54:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:54:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:54:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:54:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:55:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:55:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:55:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:55:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:55:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:55:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:56:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:56:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:56:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:56:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:56:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:56:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:57:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:57:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:57:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:57:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:57:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:57:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:58:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:58:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:58:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:58:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:58:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:58:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:59:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:59:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:59:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:59:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:59:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 18:59:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:00:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:00:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:00:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:00:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:00:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:00:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:01:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:01:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:01:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:01:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:01:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:01:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:02:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:02:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:02:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:02:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:02:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:02:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:03:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:03:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:03:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:03:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:03:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:03:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:04:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:04:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:04:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:04:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:04:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:04:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:05:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:05:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:05:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:05:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:05:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:05:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:06:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:06:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:06:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:06:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:06:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:06:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:07:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:07:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:07:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:07:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:07:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:07:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:08:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:08:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:08:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:08:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:08:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:08:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:09:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:09:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:09:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:09:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:09:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:09:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:10:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:10:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:10:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:10:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:10:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:10:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:11:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:11:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:11:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:11:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:11:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:11:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:12:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:12:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:12:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:12:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:12:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:12:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:13:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:13:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:13:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:13:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:13:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:13:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:14:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:14:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:14:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:14:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:14:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:14:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:15:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:15:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:15:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:15:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:15:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:15:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:16:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:16:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:16:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:16:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:16:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:16:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:17:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:17:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:17:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:17:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:17:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:17:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:18:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:18:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:18:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:18:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:18:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:18:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:19:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:19:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:19:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:19:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:19:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:19:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:20:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:20:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:20:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:20:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:20:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:20:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:21:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:21:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:21:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:21:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:21:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:21:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:22:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:22:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:22:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:22:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:22:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:22:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:23:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:23:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:23:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:23:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:23:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:23:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:24:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:24:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:24:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:24:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:24:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:24:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:25:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:25:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:25:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:25:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:25:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:25:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:26:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:26:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:26:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:26:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:26:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:26:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:27:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:27:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:27:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:27:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:27:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:27:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:28:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:28:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:28:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:28:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:28:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:28:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:29:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:29:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:29:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:29:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:29:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:29:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:30:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:30:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:30:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:30:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:30:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:30:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:31:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:31:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:31:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:31:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:31:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:31:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:32:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:32:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:32:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:32:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:32:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:32:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:33:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:33:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:33:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:33:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:33:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:33:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:34:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:34:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:34:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:34:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:34:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:34:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:35:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:35:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:35:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:35:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:35:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:35:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:36:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:36:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:36:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:36:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:36:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:36:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:37:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:37:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:37:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:37:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:37:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:37:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:38:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:38:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:38:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:38:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:38:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:38:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:39:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:39:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:39:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:39:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:39:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:39:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:40:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:40:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:40:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:40:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:40:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:40:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:41:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:41:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:41:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:41:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:41:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:41:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:42:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:42:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:42:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:42:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:42:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:42:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:43:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:43:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:43:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:43:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:43:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:43:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:44:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:44:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:44:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:44:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:44:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:44:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:45:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:45:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:45:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:45:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:45:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:45:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:46:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:46:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:46:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:46:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:46:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:46:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:47:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:47:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:47:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:47:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:47:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:47:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:48:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:48:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:48:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:48:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:48:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:48:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:49:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:49:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:49:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:49:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:49:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:49:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:50:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:50:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:50:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:50:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:50:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:50:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:51:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:51:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:51:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:51:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:51:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:51:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:52:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:52:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:52:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:52:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:52:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:52:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:53:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:53:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:53:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:53:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:53:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:53:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:54:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:54:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:54:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:54:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:54:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:54:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:55:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:55:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:55:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:55:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:55:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:55:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:56:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:56:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:56:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:56:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:56:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:56:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:57:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:57:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:57:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:57:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:57:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:57:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:58:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:58:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:58:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:58:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:58:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:58:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:59:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:59:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:59:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:59:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:59:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 19:59:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:00:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:00:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:00:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:00:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:00:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:00:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:01:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:01:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:01:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:01:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:01:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:01:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:02:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:02:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:02:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:02:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:02:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:02:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:03:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:03:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:03:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:03:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:03:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:03:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:04:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:04:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:04:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:04:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:04:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:04:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:05:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:05:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:05:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:05:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:05:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:05:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:06:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:06:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:06:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:06:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:06:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:06:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:07:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:07:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:07:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:07:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:07:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:07:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:08:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:08:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:08:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:08:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:08:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:08:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:09:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:09:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:09:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:09:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:09:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:09:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:10:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:10:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:10:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:10:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:10:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:10:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:11:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:11:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:11:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:11:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:11:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:11:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:12:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:12:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:12:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:12:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:12:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:12:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:13:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:13:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:13:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:13:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:13:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:13:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:14:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:14:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:14:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:14:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:14:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:14:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:15:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:15:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:15:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:15:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:15:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:15:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:16:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:16:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:16:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:16:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:16:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:16:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:17:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:17:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:17:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:17:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:17:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:17:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:18:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:18:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:18:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:18:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:18:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:18:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:19:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:19:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:19:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:19:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:19:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:19:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:20:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:20:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:20:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:20:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:20:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:20:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:21:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:21:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:21:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:21:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:21:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:21:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:22:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:22:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:22:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:22:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:22:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:22:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:23:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:23:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:23:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:23:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:23:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:23:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:24:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:24:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:24:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:24:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:24:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:24:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:25:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:25:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:25:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:25:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:25:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:25:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:26:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:26:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:26:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:26:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:26:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:26:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:27:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:27:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:27:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:27:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:27:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:27:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:28:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:28:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:28:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:28:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:28:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:28:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:29:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:29:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:29:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:29:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:29:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:29:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:30:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:30:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:30:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:30:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:30:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:30:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:31:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:31:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:31:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:31:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:31:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:31:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:32:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:32:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:32:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:32:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:32:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:32:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:33:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:33:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:33:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:33:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:33:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:33:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:34:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:34:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:34:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:34:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:34:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:34:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:35:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:35:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:35:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:35:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:35:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:35:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:36:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:36:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:36:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:36:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:36:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:36:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:37:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:37:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:37:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:37:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:37:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:37:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:38:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:38:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:38:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:38:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:38:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:38:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:39:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:39:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:39:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:39:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:39:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:39:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:40:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:40:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:40:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:40:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:40:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:40:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:41:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:41:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:41:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:41:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:41:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:41:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:42:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:42:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:42:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:42:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:42:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:42:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:43:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:43:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:43:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:43:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:43:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:43:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:44:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:44:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:44:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:44:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:44:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:44:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:45:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:45:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:45:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:45:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:45:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:45:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:46:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:46:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:46:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:46:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:46:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:46:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:47:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:47:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:47:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:47:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:47:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:47:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:48:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:48:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:48:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:48:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:48:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:48:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:49:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:49:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:49:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:49:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:49:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:49:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:50:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:50:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:50:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:50:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:50:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:50:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:51:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:51:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:51:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:51:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:51:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:51:51 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:52:01 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:52:11 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:52:21 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:52:31 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO 04-27 20:52:41 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%. INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK INFO: 10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK |