logo

InferX AI Function Platform (Lambda Function for Inference)

    --   Serve tens models in one box with ultra-fast (<2 sec) cold start (contact: support@inferx.net)

funcpod

tenant namespace podname
public Qwen public/Qwen/Qwen2.5-Math-1.5B/132/928

state

state time
Init 2025/03/12 04:03:35
Creating 2025/03/12 04:03:35
Standby 2025/03/12 04:03:39
Standby 2025/03/12 04:03:39
Resuming 2025/03/13 05:23:24
Ready 2025/03/13 05:23:24

log

Exception in thread Thread-2 (_report_usage_worker):

Traceback (most recent call last):

File "/usr/lib/python3.12/threading.py", line 1075, in _bootstrap_inner

INFO 03-12 22:23:24 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

 self.run()

File "/usr/lib/python3.12/threading.py", line 1012, in run

 self._target(*self._args, **self._kwargs)

File "/usr/local/lib/python3.12/dist-packages/vllm/usage/usage_lib.py", line 149, in _report_usage_worker

 self._report_continous_usage()

File "/usr/local/lib/python3.12/dist-packages/vllm/usage/usage_lib.py", line 197, in _report_continous_usage

 time.sleep(600)

OSError: [Errno 85] Interrupted system call should be restarted

INFO 03-12 22:23:24 logger.py:36] Received request cmpl-c912145a4d784822918448c3b1a2ea35-0: prompt: 'Find the value of $x$ that satisfies the equation $4x+5 = 6x+7$.', params: SamplingParams(n=1, best_of=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.0, top_p=1.0, top_k=-1, min_p=0.0, seed=None, use_beam_search=False, length_penalty=1.0, early_stopping=False, stop=[], stop_token_ids=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=1000, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None), prompt_token_ids: [9885, 279, 897, 315, 400, 87, 3, 429, 67901, 279, 23606, 400, 19, 87, 10, 20, 284, 220, 21, 87, 10, 22, 12947], lora_request: None, prompt_adapter_request: None.

INFO:  10.1.2.3:123 - "POST /v1/completions HTTP/1.1" 200 OK

INFO 03-12 22:23:24 engine.py:288] Added request cmpl-c912145a4d784822918448c3b1a2ea35-0.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:23:38 metrics.py:351] Avg prompt throughput: 1.6 tokens/s, Avg generation throughput: 25.6 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:23:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:23:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:24:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:24:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:24:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:24:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:24:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:24:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:25:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:25:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:25:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:25:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:25:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:25:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:26:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:26:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:26:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:26:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:26:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:26:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:27:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:27:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:27:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:27:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:27:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:27:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:28:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:28:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:28:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:28:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:28:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:28:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:29:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:29:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:29:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:29:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:29:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:29:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:30:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:30:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:30:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:30:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:30:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:30:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:31:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:31:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:31:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:31:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:31:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:31:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:32:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:32:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:32:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:32:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:32:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:32:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:33:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:33:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:33:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:33:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:33:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:33:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:34:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:34:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:34:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:34:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:34:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:34:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:35:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:35:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:35:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:35:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:35:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:35:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:36:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:36:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:36:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:36:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:36:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:36:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:37:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:37:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:37:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:37:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:37:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:37:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:38:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:38:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:38:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:38:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:38:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:38:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:39:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:39:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:39:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:39:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:39:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:39:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:40:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:40:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:40:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:40:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:40:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:40:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:41:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:41:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:41:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:41:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:41:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:41:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:42:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:42:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:42:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:42:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:42:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:42:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:43:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:43:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:43:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:43:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:43:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:43:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:44:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:44:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:44:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:44:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:44:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:44:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:45:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:45:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:45:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:45:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:45:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:45:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:46:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:46:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:46:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:46:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:46:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:46:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:47:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:47:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:47:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:47:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:47:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:47:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:48:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:48:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:48:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:48:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:48:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:48:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:49:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:49:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:49:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:49:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:49:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:49:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:50:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:50:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:50:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:50:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:50:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:50:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:51:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:51:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:51:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:51:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:51:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:51:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:52:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:52:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:52:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:52:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:52:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:52:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:53:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:53:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:53:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:53:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:53:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:53:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:54:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:54:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:54:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:54:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:54:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:54:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:55:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:55:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:55:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:55:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:55:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:55:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:56:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:56:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:56:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:56:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:56:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:56:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:57:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:57:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:57:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:57:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:57:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:57:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:58:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:58:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:58:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:58:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:58:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:58:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:59:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:59:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:59:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:59:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:59:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 22:59:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:00:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:00:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:00:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:00:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:00:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:00:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:01:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:01:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:01:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:01:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:01:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:01:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:02:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:02:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:02:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:02:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:02:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:02:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:03:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:03:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:03:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:03:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:03:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:03:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:04:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:04:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:04:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:04:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:04:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:04:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:05:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:05:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:05:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:05:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:05:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:05:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:06:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:06:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:06:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:06:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:06:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:06:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:07:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:07:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:07:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:07:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:07:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:07:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:08:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:08:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:08:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:08:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:08:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:08:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:09:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:09:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:09:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:09:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:09:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:09:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:10:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:10:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:10:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:10:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:10:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:10:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:11:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:11:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:11:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:11:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:11:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:11:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:12:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:12:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:12:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:12:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:12:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:12:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:13:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:13:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:13:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:13:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:13:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:13:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:14:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:14:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:14:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:14:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:14:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:14:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:15:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:15:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:15:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:15:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:15:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:15:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:16:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:16:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:16:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:16:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:16:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:16:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:17:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:17:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:17:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:17:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:17:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:17:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:18:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:18:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:18:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:18:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:18:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:18:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:19:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:19:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:19:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:19:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:19:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:19:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:20:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:20:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:20:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:20:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:20:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:20:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:21:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:21:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:21:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:21:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:21:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:21:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:22:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:22:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:22:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:22:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:22:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:22:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:23:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:23:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:23:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:23:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:23:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:23:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:24:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:24:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:24:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:24:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:24:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:24:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:25:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:25:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:25:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:25:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:25:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:25:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:26:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:26:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:26:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:26:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:26:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:26:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:27:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:27:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:27:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:27:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:27:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:27:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:28:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:28:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:28:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:28:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:28:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:28:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:29:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:29:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:29:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:29:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:29:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:29:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:30:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:30:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:30:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:30:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:30:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:30:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:31:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:31:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:31:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:31:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:31:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:31:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:32:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:32:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:32:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:32:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:32:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:32:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:33:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:33:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:33:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:33:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:33:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:33:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:34:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:34:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:34:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:34:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:34:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:34:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:35:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:35:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:35:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:35:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:35:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:35:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:36:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:36:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:36:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:36:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:36:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:36:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:37:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:37:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:37:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:37:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:37:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:37:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:38:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:38:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:38:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:38:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:38:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:38:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:39:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:39:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:39:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:39:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:39:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:39:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:40:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:40:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:40:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:40:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:40:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:40:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:41:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:41:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:41:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:41:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:41:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:41:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:42:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:42:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:42:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:42:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:42:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:42:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:43:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:43:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:43:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:43:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:43:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:43:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:44:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:44:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:44:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:44:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:44:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:44:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:45:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:45:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:45:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:45:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:45:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:45:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:46:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:46:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:46:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:46:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:46:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:46:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:47:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:47:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:47:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:47:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:47:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:47:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:48:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:48:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:48:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:48:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:48:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:48:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:49:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:49:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:49:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:49:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:49:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:49:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:50:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:50:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:50:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:50:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:50:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:50:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:51:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:51:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:51:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:51:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:51:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:51:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:52:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:52:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:52:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:52:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:52:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:52:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:53:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:53:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:53:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:53:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:53:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:53:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:54:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:54:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:54:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:54:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:54:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:54:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:55:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:55:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:55:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:55:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:55:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:55:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:56:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:56:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:56:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:56:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:56:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:56:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:57:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:57:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:57:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:57:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:57:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:57:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:58:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:58:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:58:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:58:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:58:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:58:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:59:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:59:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:59:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:59:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:59:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-12 23:59:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:00:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:00:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:00:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:00:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:00:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:00:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:01:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:01:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:01:28 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:01:38 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:01:48 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:01:58 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:02:08 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:02:18 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:02:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:02:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:02:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:02:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:03:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:03:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:03:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:03:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:03:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:03:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:04:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:04:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:04:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:04:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:04:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:04:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:05:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:05:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:05:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:05:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:05:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:05:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:06:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:06:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:06:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:06:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:06:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:06:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:07:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:07:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:07:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:07:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:07:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:07:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:08:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:08:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:08:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:08:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:08:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:08:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:09:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:09:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:09:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:09:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:09:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:09:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:10:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:10:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:10:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:10:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:10:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:10:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:11:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:11:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:11:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:11:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:11:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:11:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:12:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:12:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:12:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:12:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:12:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:12:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:13:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:13:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:13:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:13:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:13:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:13:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:14:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:14:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:14:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:14:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:14:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:14:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:15:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:15:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:15:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:15:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:15:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:15:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:16:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:16:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:16:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:16:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:16:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:16:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:17:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:17:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:17:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:17:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:17:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:17:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:18:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:18:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:18:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:18:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:18:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:18:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:19:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:19:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:19:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:19:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:19:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:19:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:20:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:20:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:20:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:20:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:20:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:20:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:21:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:21:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:21:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:21:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:21:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:21:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:22:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:22:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:22:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:22:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:22:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:22:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:23:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:23:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:23:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:23:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:23:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:23:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:24:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:24:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:24:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:24:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:24:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:24:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:25:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:25:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:25:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:25:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:25:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:25:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:26:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:26:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:26:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:26:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:26:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:26:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:27:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:27:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:27:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:27:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:27:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:27:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:28:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:28:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:28:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:28:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:28:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:28:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:29:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:29:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:29:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:29:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:29:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:29:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:30:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:30:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:30:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:30:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:30:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:30:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:31:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:31:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:31:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:31:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:31:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:31:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:32:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:32:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:32:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:32:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:32:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:32:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:33:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:33:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:33:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:33:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:33:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:33:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:34:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:34:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:34:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:34:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:34:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:34:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:35:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:35:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:35:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:35:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:35:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:35:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:36:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:36:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:36:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:36:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:36:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:36:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:37:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:37:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:37:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:37:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:37:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:37:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:38:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:38:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:38:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:38:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:38:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:38:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:39:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:39:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:39:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:39:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:39:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:39:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:40:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:40:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:40:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:40:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:40:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:40:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:41:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:41:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:41:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:41:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:41:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:41:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:42:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:42:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:42:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:42:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:42:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:42:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:43:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:43:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:43:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:43:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:43:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:43:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:44:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:44:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:44:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:44:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:44:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:44:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:45:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:45:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:45:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:45:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:45:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:45:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:46:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:46:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:46:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:46:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:46:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:46:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:47:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:47:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:47:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:47:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:47:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:47:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:48:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:48:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:48:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:48:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:48:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:48:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:49:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:49:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:49:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:49:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:49:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:49:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:50:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:50:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:50:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:50:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:50:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:50:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:51:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:51:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:51:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:51:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:51:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:51:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:52:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:52:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:52:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:52:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:52:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:52:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:53:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:53:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:53:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:53:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:53:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:53:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:54:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:54:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:54:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:54:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:54:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:54:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:55:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:55:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:55:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:55:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:55:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:55:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:56:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:56:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:56:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:56:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:56:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:56:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:57:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:57:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:57:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:57:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:57:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:57:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:58:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:58:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:58:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:58:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:58:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:58:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:59:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:59:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:59:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:59:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:59:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 00:59:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:00:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:00:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:00:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:00:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:00:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:00:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:01:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:01:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:01:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:01:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:01:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:01:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:02:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:02:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:02:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:02:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:02:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:02:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:03:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:03:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:03:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:03:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:03:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:03:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:04:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:04:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:04:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:04:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:04:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:04:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:05:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:05:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:05:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:05:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:05:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:05:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:06:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:06:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:06:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:06:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:06:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:06:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:07:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:07:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:07:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:07:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:07:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:07:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:08:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:08:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:08:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:08:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:08:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:08:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:09:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:09:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:09:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:09:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:09:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:09:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:10:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:10:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:10:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:10:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:10:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:10:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:11:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:11:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:11:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:11:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:11:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:11:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:12:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:12:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:12:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:12:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:12:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:12:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:13:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:13:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:13:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:13:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:13:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:13:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:14:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:14:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:14:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:14:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:14:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:14:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:15:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:15:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:15:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:15:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:15:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:15:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:16:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:16:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:16:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:16:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:16:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:16:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:17:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:17:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:17:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:17:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:17:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:17:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:18:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:18:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:18:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:18:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:18:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:18:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:19:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:19:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:19:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:19:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:19:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:19:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:20:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:20:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:20:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:20:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:20:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:20:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:21:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:21:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:21:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:21:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:21:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:21:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:22:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:22:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:22:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:22:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:22:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:22:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:23:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:23:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:23:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:23:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:23:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:23:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:24:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:24:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:24:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:24:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:24:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:24:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:25:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:25:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:25:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:25:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:25:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:25:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:26:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:26:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:26:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:26:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:26:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:26:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:27:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:27:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:27:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:27:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:27:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:27:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:28:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:28:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:28:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:28:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:28:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:28:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:29:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:29:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:29:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:29:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:29:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:29:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:30:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:30:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:30:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:30:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:30:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:30:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:31:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:31:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:31:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:31:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:31:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:31:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:32:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:32:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:32:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:32:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:32:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:32:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:33:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:33:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:33:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:33:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:33:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:33:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:34:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:34:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:34:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:34:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:34:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:34:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:35:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:35:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:35:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:35:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:35:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:35:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:36:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:36:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:36:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:36:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:36:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:36:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:37:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:37:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:37:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:37:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:37:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:37:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:38:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:38:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:38:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:38:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:38:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:38:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:39:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:39:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:39:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:39:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:39:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:39:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:40:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:40:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:40:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:40:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:40:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:40:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:41:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:41:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:41:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:41:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:41:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:41:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:42:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:42:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:42:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:42:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:42:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:42:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:43:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:43:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:43:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:43:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:43:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:43:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:44:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:44:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:44:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:44:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:44:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:44:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:45:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:45:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:45:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:45:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:45:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:45:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:46:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:46:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:46:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:46:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:46:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:46:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:47:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:47:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:47:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:47:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:47:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:47:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:48:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:48:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:48:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:48:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:48:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:48:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:49:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:49:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:49:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:49:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:49:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:49:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:50:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:50:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:50:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:50:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:50:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:50:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:51:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:51:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:51:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:51:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:51:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:51:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:52:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:52:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:52:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:52:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:52:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:52:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:53:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:53:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:53:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:53:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:53:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:53:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:54:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:54:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:54:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:54:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:54:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:54:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:55:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:55:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:55:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:55:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:55:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:55:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:56:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:56:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:56:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:56:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:56:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:56:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:57:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:57:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:57:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:57:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:57:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:57:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:58:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:58:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:58:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:58:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:58:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:58:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:59:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:59:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:59:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:59:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:59:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 01:59:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:00:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:00:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:00:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:00:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:00:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:00:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:01:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:01:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:01:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:01:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:01:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:01:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:02:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:02:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:02:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:02:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:02:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:02:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:03:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:03:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:03:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:03:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:03:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:03:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:04:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:04:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:04:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:04:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:04:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:04:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:05:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:05:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:05:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:05:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:05:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:05:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:06:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:06:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:06:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:06:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:06:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:06:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:07:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:07:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:07:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:07:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:07:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:07:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:08:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:08:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:08:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:08:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:08:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:08:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:09:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:09:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:09:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:09:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:09:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:09:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:10:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:10:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:10:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:10:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:10:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:10:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:11:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:11:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:11:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:11:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:11:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:11:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:12:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:12:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:12:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:12:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:12:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:12:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:13:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:13:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:13:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:13:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:13:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:13:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:14:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:14:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:14:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:14:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:14:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:14:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:15:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:15:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:15:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:15:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:15:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:15:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:16:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:16:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:16:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:16:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:16:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:16:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:17:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:17:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:17:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:17:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:17:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:17:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:18:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:18:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:18:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:18:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:18:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:18:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:19:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:19:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:19:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:19:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:19:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:19:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:20:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:20:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:20:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:20:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:20:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:20:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:21:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:21:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:21:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:21:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:21:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:21:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:22:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:22:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:22:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:22:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:22:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:22:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:23:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:23:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:23:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:23:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:23:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:23:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:24:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:24:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:24:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:24:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:24:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:24:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:25:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:25:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:25:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:25:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:25:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:25:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:26:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:26:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:26:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:26:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:26:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:26:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:27:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:27:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:27:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:27:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:27:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:27:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:28:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:28:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:28:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:28:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:28:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:28:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:29:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:29:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:29:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:29:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:29:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:29:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:30:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:30:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:30:29 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:30:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:30:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:30:59 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:31:09 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO 03-13 02:31:19 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK

INFO:  10.1.2.3:123 - "GET /health HTTP/1.1" 200 OK