Inference stop working after running for a while

[liyimeng]

Describe the bug
I deploy using oneclick.sh all work expected at beginning, but it failed after have some conversations.

To Reproduce
Steps to reproduce the behavior:

1. Deploy with oneclick.sh
2. Start a call
3. After running for a while, I get error "Failed to initialised model" web
4. See attached logs for backend errors

Expected behavior
Stably running, but not fail when time past.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

cpp server logs

[init_sys_prompt] 检测到残留的 is_breaking=True，重置为 False
[清空输出] 已清空 /llama.cpp-omni/tools/omni/output_9060 下的子文件夹内容 (删除 0 项)
[极速恢复] 调用 C++ update_session_config 重置状态: {"media_type": 2, "duplex_mode": false, "language": "en", "voice_audio": "/app/WebRTC_Demo/cpp_server/assets/default_ref_audio.wav"}
Traceback (most recent call last):
  File "/app/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 101, in map_httpcore_exceptions
    yield
  File "/app/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 394, in handle_async_request
    resp = await self._pool.handle_async_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 256, in handle_async_request
    raise exc from None
  File "/app/.venv/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 236, in handle_async_request
    response = await connection.handle_async_request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/httpcore/_async/connection.py", line 101, in handle_async_request
    raise exc
  File "/app/.venv/lib/python3.12/site-packages/httpcore/_async/connection.py", line 78, in handle_async_request
    stream = await self._connect(request)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/httpcore/_async/connection.py", line 124, in _connect
    stream = await self._network_backend.connect_tcp(**kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/httpcore/_backends/auto.py", line 31, in connect_tcp
    return await self._backend.connect_tcp(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/httpcore/_backends/anyio.py", line 113, in connect_tcp
    with map_exceptions(exc_map):
  File "/usr/lib/python3.12/contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "/app/.venv/lib/python3.12/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
    raise to_exc(exc) from exc
httpcore.ConnectError: All connection attempts failed

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/WebRTC_Demo/cpp_server/minicpmo_cpp_http_server.py", line 1208, in init_sys_prompt
    resp = await http_client.post(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1859, in post
    return await self.request(
           ^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1540, in request
    return await self.send(request, auth=auth, follow_redirects=follow_redirects)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1629, in send
    response = await self._send_handling_auth(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1657, in _send_handling_auth
    response = await self._send_handling_redirects(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1694, in _send_handling_redirects
    response = await self._send_single_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1730, in _send_single_request
    response = await transport.handle_async_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 393, in handle_async_request
    with map_httpcore_exceptions():
  File "/usr/lib/python3.12/contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "/app/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 118, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ConnectError: All connection attempts failed
INFO:     172.25.4.56:44376 - "POST /omni/init_sys_prompt HTTP/1.1" 500 Internal Server Error
======= [独立线程] 收到快速停止指令 =======
[独立线程] is_breaking 已设置为 True (stop)
[独立线程] C++ break 调用异常 (stop): HTTPConnectionPool(host='127.0.0.1', port=19060): Max retries exceeded with url: /v1/stream/break (Caused by NewConnectionError("HTTPConnection(host='127.0.0.1', port=19060): Failed to establish a new connection: [Errno 111] Connection refused"))

[tc-mb]

Hi @liyimeng, thanks for the detailed report.

The traceback you posted is from the Python HTTP server failing to connect to 127.0.0.1:19060, which is the C++ inference server (llama-server). That means the C++ backend has already exited / crashed by the time this error appears, so the root cause needs to be found in the C++ server's own log, not this Python traceback.

Could you help us collect a bit more info so we can pin this down?

1. C++ server log: under your deployment dir, please grab the full output from cpp_server/logs/llama_server*.log (or wherever oneclick.sh redirects it). The last few hundred lines around the failure are most useful — we need to see whether it's CUDA OOM, an assert in token2wav, a segfault, etc.
2. GPU memory trend: while reproducing, please run watch -n 1 nvidia-smi in another terminal. Does VRAM usage keep climbing across conversations? If yes it's likely a session-state leak we need to fix.
3. Process status after the failure: ps -ef | grep llama-server — is the process actually gone, or zombie / hung?
4. Reproduction trigger: roughly how many turns / minutes before it dies? Does it correlate with a specific kind of input (long audio, video, language switching, etc.)?
5. Environment: GPU model, CUDA / driver version, Docker or bare metal, and which branch / commit you're on.

Once we have the C++ side log we can give you a concrete fix or workaround.

FYI, we're also working on a new demo with cleaner setup and better long-running stability, which we'll release soon — will follow up here once it's out. Thanks!